Method for selecting appropriate sentence from documents on the WWW for the open-ended conversation dialog system

Masahiro Shibata, Yoichi Tomiura, Tomomi Nishiguchi

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

We propose an open-ended dialog system that generates a proper sentence to a user's utterance using abundant documents on the World Wide Web as sources. Existing knowledge-based dialog systems give meaningful information to a user, but they are unsuitable for open-ended input. The system Eliza can handle open-ended input, but it gives no meaningful information. Our system lies between the above two dialog systems; it converses on various topics and gives meaningful information related to the user's utterances. The system selects an appropriate sentence as a response from documents gathered through the Web, on the basis of surface cohesion and shallow semantic coherence. The surface cohesion follows centering theory and the semantic coherence is calculated on the basis of the conditional distribution and inverse document frequency of content words (nouns, verbs, and adjectives.) We developed a trial system to converse about movies and experimentally found that the proposed method generated 66 % appropriate responses.

Original languageEnglish
Pages (from-to)507-519
Number of pages13
JournalTransactions of the Japanese Society for Artificial Intelligence
Volume24
Issue number6
DOIs
Publication statusPublished - Jan 1 2009

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Method for selecting appropriate sentence from documents on the WWW for the open-ended conversation dialog system'. Together they form a unique fingerprint.

  • Cite this