Context-Aware Latent Dirichlet Allocation for Topic Segmentation

研究成果: Chapter in Book/Report/Conference proceedingConference contribution

抜粋

We propose a new generative model for topic segmentation based on Latent Dirichlet Allocation. The task is to divide a document into a sequence of topically coherent segments, while preserving long topic change-points (coherency) and keeping short topic segments from getting merged (saliency). Most of the existing models either fuse topic segments by keywords or focus on modeling word co-occurrence patterns without merging. They can hardly achieve both coherency and saliency since many words have high uncertainties in topic assignments due to their polysemous nature. To solve this problem, we introduce topic-specific co-occurrence of word pairs within contexts in modeling, to generate more coherent segments and alleviate the influence of irrelevant words on topic assignment. We also design an optimization algorithm to eliminate redundant items in the generated topic segments. Experimental results show that our proposal produces significant improvements in both topic coherence and topic segmentation.

元の言語英語
ホスト出版物のタイトルAdvances in Knowledge Discovery and Data Mining - 24th Pacific-Asia Conference, PAKDD 2020, Proceedings
編集者Hady W. Lauw, Ee-Peng Lim, Raymond Chi-Wing Wong, Alexandros Ntoulas, See-Kiong Ng, Sinno Jialin Pan
出版者Springer
ページ475-486
ページ数12
ISBN(印刷物)9783030474256
DOI
出版物ステータス出版済み - 1 1 2020
イベント24th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2020 - Singapore, シンガポール
継続期間: 5 11 20205 14 2020

出版物シリーズ

名前Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
12084 LNAI
ISSN(印刷物)0302-9743
ISSN(電子版)1611-3349

会議

会議24th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2020
シンガポール
Singapore
期間5/11/205/14/20

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

フィンガープリント Context-Aware Latent Dirichlet Allocation for Topic Segmentation' の研究トピックを掘り下げます。これらはともに一意のフィンガープリントを構成します。

  • これを引用

    Li, W., Matsukawa, T., Saigo, H., & Suzuki, E. (2020). Context-Aware Latent Dirichlet Allocation for Topic Segmentation. : H. W. Lauw, E-P. Lim, R. C-W. Wong, A. Ntoulas, S-K. Ng, & S. J. Pan (版), Advances in Knowledge Discovery and Data Mining - 24th Pacific-Asia Conference, PAKDD 2020, Proceedings (pp. 475-486). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); 巻数 12084 LNAI). Springer. https://doi.org/10.1007/978-3-030-47426-3_37