Extraction of relevant snippets from web pages using hybrid features

Jun Zeng, Junhao Wen, Qingyu Xiong, Sachio Hirokawa

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    As the amount of web pages increase, identifying and retrieving distinct contents from the web has increasingly become more and more difficult. The traditional approach for extracting data from web page documents is to analyze the DOM (Document Object Model) structure of a HTML page and find a common pattern. However, the number of possible DOM layout patterns is virtually infinite, which means that there is no common pattern that can be used for all kinds of web pages. In this paper, we focus on the pages that are linked to a search engine and aim to analyze the features of relevant and meaningful contents instead of a common pattern. Three features of relevant snippets are introduced. They are: quantity of text, correlation between snippet and query that is inputted into a search engine, and HTML structure. Nine parameters are used to describe the three features. Also, a SVM learning experiment is conducted to verify the effectiveness of the three features. The results show that the HTML structure feature is the most effective feature which can determine whether a snippet is relevant or not.

    Original languageEnglish
    Title of host publicationProceedings of the 2012 IIAI International Conference on Advanced Applied Informatics, IIAIAAI 2012
    Pages209-213
    Number of pages5
    DOIs
    Publication statusPublished - Dec 14 2012
    Event1st IIAI International Conference on Advanced Applied Informatics, IIAIAAI 2012 - Fukuoka, Japan
    Duration: Sep 20 2012Sep 22 2012

    Publication series

    NameProceedings of the 2012 IIAI International Conference on Advanced Applied Informatics, IIAIAAI 2012

    Other

    Other1st IIAI International Conference on Advanced Applied Informatics, IIAIAAI 2012
    Country/TerritoryJapan
    CityFukuoka
    Period9/20/129/22/12

    All Science Journal Classification (ASJC) codes

    • Information Systems

    Fingerprint

    Dive into the research topics of 'Extraction of relevant snippets from web pages using hybrid features'. Together they form a unique fingerprint.

    Cite this