LTDE: A layout tree based approach for deep page data extraction

Jun Zeng, Feng Li, Brendan Flanagan, Sachio Hirokawa

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Content extraction from deep Web pages has received great attention in recent years. However, the increasingly complicated HTML structure of Web documents makes it more difficult to recognize the data records by only analyzing the HTML source code. In this paper, we propose a method named LTDE to extract data records from a deepWeb page. Instead of analyzing the HTML source code, LTDE utilizes the visual features of data records in deep Web pages. A Web page is considered as a finite set of visual blocks. The data records are the visual blocks that have similar layout. We also propose a pattern recognizing method named layout tree to cluster the similar layout visual blocks. The weight of all clusters is calculated, and the visual blocks in the cluster that has the highest weight are chosen as the data records to be extracted. The experiment results show that LTDE has higher effectiveness and better robustness for Web data extraction compared to previous works.

Original languageEnglish
Pages (from-to)1067-1078
Number of pages12
JournalIEICE Transactions on Information and Systems
VolumeE100D
Issue number5
DOIs
Publication statusPublished - May 2017

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'LTDE: A layout tree based approach for deep page data extraction'. Together they form a unique fingerprint.

  • Cite this