Mining peculiar compositions of frequent substrings from sparse text data using background texts

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

We consider mining unusual patterns from text T. Unlike existing methods which assume probabilistic models and use simple estimation methods, we employ a set B of background text in addition to T and compositions w = xy of x and y as patterns. A string w is peculiar if there exist x and y such that w = xy, each of x and y is more frequent in B than in T, and conversely w = xy is more frequent in T. The frequency of xy in T is very small since x and y are infrequent in T, but xy is relatively abundant in T compared to xy in B. Despite these complex conditions for peculiar compositions, we develop a fast algorithm to find peculiar compositions using the suffix tree. Experiments using DNA sequences show scalability of our algorithm due to our pruning techniques and the superiority of the concept of the peculiar composition.

Original languageEnglish
Title of host publicationMachine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings
Pages596-611
Number of pages16
EditionPART 1
DOIs
Publication statusPublished - Nov 9 2009
EventEuropean Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2009 - Bled, Slovenia
Duration: Sep 7 2009Sep 11 2009

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 1
Volume5781 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherEuropean Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2009
CountrySlovenia
CityBled
Period9/7/099/11/09

Fingerprint

Mining
Chemical analysis
Suffix Tree
DNA sequences
Pruning
Probabilistic Model
DNA Sequence
Fast Algorithm
Scalability
Strings
Text
Background
Experiment
Experiments

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Ikeda, D., & Suzuki, E. (2009). Mining peculiar compositions of frequent substrings from sparse text data using background texts. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings (PART 1 ed., pp. 596-611). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5781 LNAI, No. PART 1). https://doi.org/10.1007/978-3-642-04180-8_56

Mining peculiar compositions of frequent substrings from sparse text data using background texts. / Ikeda, Daisuke; Suzuki, Einoshin.

Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings. PART 1. ed. 2009. p. 596-611 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5781 LNAI, No. PART 1).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ikeda, D & Suzuki, E 2009, Mining peculiar compositions of frequent substrings from sparse text data using background texts. in Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings. PART 1 edn, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 1, vol. 5781 LNAI, pp. 596-611, European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2009, Bled, Slovenia, 9/7/09. https://doi.org/10.1007/978-3-642-04180-8_56
Ikeda D, Suzuki E. Mining peculiar compositions of frequent substrings from sparse text data using background texts. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings. PART 1 ed. 2009. p. 596-611. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1). https://doi.org/10.1007/978-3-642-04180-8_56
Ikeda, Daisuke ; Suzuki, Einoshin. / Mining peculiar compositions of frequent substrings from sparse text data using background texts. Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings. PART 1. ed. 2009. pp. 596-611 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1).
@inproceedings{5c47a6d3818547cca0360f5c32db1dfe,
title = "Mining peculiar compositions of frequent substrings from sparse text data using background texts",
abstract = "We consider mining unusual patterns from text T. Unlike existing methods which assume probabilistic models and use simple estimation methods, we employ a set B of background text in addition to T and compositions w = xy of x and y as patterns. A string w is peculiar if there exist x and y such that w = xy, each of x and y is more frequent in B than in T, and conversely w = xy is more frequent in T. The frequency of xy in T is very small since x and y are infrequent in T, but xy is relatively abundant in T compared to xy in B. Despite these complex conditions for peculiar compositions, we develop a fast algorithm to find peculiar compositions using the suffix tree. Experiments using DNA sequences show scalability of our algorithm due to our pruning techniques and the superiority of the concept of the peculiar composition.",
author = "Daisuke Ikeda and Einoshin Suzuki",
year = "2009",
month = "11",
day = "9",
doi = "10.1007/978-3-642-04180-8_56",
language = "English",
isbn = "3642041795",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
number = "PART 1",
pages = "596--611",
booktitle = "Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings",
edition = "PART 1",

}

TY - GEN

T1 - Mining peculiar compositions of frequent substrings from sparse text data using background texts

AU - Ikeda, Daisuke

AU - Suzuki, Einoshin

PY - 2009/11/9

Y1 - 2009/11/9

N2 - We consider mining unusual patterns from text T. Unlike existing methods which assume probabilistic models and use simple estimation methods, we employ a set B of background text in addition to T and compositions w = xy of x and y as patterns. A string w is peculiar if there exist x and y such that w = xy, each of x and y is more frequent in B than in T, and conversely w = xy is more frequent in T. The frequency of xy in T is very small since x and y are infrequent in T, but xy is relatively abundant in T compared to xy in B. Despite these complex conditions for peculiar compositions, we develop a fast algorithm to find peculiar compositions using the suffix tree. Experiments using DNA sequences show scalability of our algorithm due to our pruning techniques and the superiority of the concept of the peculiar composition.

AB - We consider mining unusual patterns from text T. Unlike existing methods which assume probabilistic models and use simple estimation methods, we employ a set B of background text in addition to T and compositions w = xy of x and y as patterns. A string w is peculiar if there exist x and y such that w = xy, each of x and y is more frequent in B than in T, and conversely w = xy is more frequent in T. The frequency of xy in T is very small since x and y are infrequent in T, but xy is relatively abundant in T compared to xy in B. Despite these complex conditions for peculiar compositions, we develop a fast algorithm to find peculiar compositions using the suffix tree. Experiments using DNA sequences show scalability of our algorithm due to our pruning techniques and the superiority of the concept of the peculiar composition.

UR - http://www.scopus.com/inward/record.url?scp=70350633754&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70350633754&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-04180-8_56

DO - 10.1007/978-3-642-04180-8_56

M3 - Conference contribution

AN - SCOPUS:70350633754

SN - 3642041795

SN - 9783642041792

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 596

EP - 611

BT - Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2009, Proceedings

ER -