Extraction of relevant snippets from web pages using hybrid features

Jun Zeng, Junhao Wen, Qingyu Xiong, Sachio Hirokawa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

As the amount of web pages increase, identifying and retrieving distinct contents from the web has increasingly become more and more difficult. The traditional approach for extracting data from web page documents is to analyze the DOM (Document Object Model) structure of a HTML page and find a common pattern. However, the number of possible DOM layout patterns is virtually infinite, which means that there is no common pattern that can be used for all kinds of web pages. In this paper, we focus on the pages that are linked to a search engine and aim to analyze the features of relevant and meaningful contents instead of a common pattern. Three features of relevant snippets are introduced. They are: quantity of text, correlation between snippet and query that is inputted into a search engine, and HTML structure. Nine parameters are used to describe the three features. Also, a SVM learning experiment is conducted to verify the effectiveness of the three features. The results show that the HTML structure feature is the most effective feature which can determine whether a snippet is relevant or not.

Original languageEnglish
Title of host publicationProceedings of the 2012 IIAI International Conference on Advanced Applied Informatics, IIAIAAI 2012
Pages209-213
Number of pages5
DOIs
Publication statusPublished - Dec 14 2012
Event1st IIAI International Conference on Advanced Applied Informatics, IIAIAAI 2012 - Fukuoka, Japan
Duration: Sep 20 2012Sep 22 2012

Publication series

NameProceedings of the 2012 IIAI International Conference on Advanced Applied Informatics, IIAIAAI 2012

Other

Other1st IIAI International Conference on Advanced Applied Informatics, IIAIAAI 2012
CountryJapan
CityFukuoka
Period9/20/129/22/12

All Science Journal Classification (ASJC) codes

  • Information Systems

Fingerprint Dive into the research topics of 'Extraction of relevant snippets from web pages using hybrid features'. Together they form a unique fingerprint.

  • Cite this

    Zeng, J., Wen, J., Xiong, Q., & Hirokawa, S. (2012). Extraction of relevant snippets from web pages using hybrid features. In Proceedings of the 2012 IIAI International Conference on Advanced Applied Informatics, IIAIAAI 2012 (pp. 209-213). [6337190] (Proceedings of the 2012 IIAI International Conference on Advanced Applied Informatics, IIAIAAI 2012). https://doi.org/10.1109/IIAI-AAI.2012.50