Mining peculiar compositions of frequent substrings from sparse text data using background texts

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

We consider mining unusual patterns from text T. Unlike existing methods which assume probabilistic models and use simple estimation methods, we employ a set B of background text in addition to T and compositions w = xy of x and y as patterns. A string w is peculiar if there exist x and y such that w = xy, each of x and y is more frequent in B than in T, and conversely w = xy is more frequent in T. The frequency of xy in T is very small since x and y are infrequent in T, but xy is relatively abundant in T compared to xy in B. Despite these complex conditions for peculiar compositions, we develop a fast algorithm to find peculiar compositions using the suffix tree. Experiments using DNA sequences show scalability of our algorithm due to our pruning techniques and the superiority of the concept of the peculiar composition.

Original languageEnglish
Title of host publicationMachine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings
Pages596-611
Number of pages16
EditionPART 1
DOIs
Publication statusPublished - 2009
EventEuropean Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2009 - Bled, Slovenia
Duration: Sep 7 2009Sep 11 2009

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 1
Volume5781 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherEuropean Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2009
CountrySlovenia
CityBled
Period9/7/099/11/09

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Mining peculiar compositions of frequent substrings from sparse text data using background texts'. Together they form a unique fingerprint.

Cite this