Lyric video analysis using text detection and tracking

Shota Sakaguchi, Jun Kato, Masataka Goto, Seiichi Uchida

研究成果: Chapter in Book/Report/Conference proceedingConference contribution

抄録

We attempt to recognize and track lyric words in lyric videos. Lyric video is a music video showing the lyric words of a song. The main characteristic of lyric videos is that the lyric words are shown at frames synchronously with the music. The difficulty of recognizing and tracking the lyric words is that (1) the words are often decorated and geometrically distorted and (2) the words move arbitrarily and drastically in the video frame. The purpose of this paper is to analyze the motion of the lyric words in lyric videos, as the first step of automatic lyric video generation. In order to analyze the motion of lyric words, we first apply a state-of-the-art scene text detector and recognizer to each video frame. Then, lyric-frame matching is performed to establish the optimal correspondence between lyric words and the frames. After fixing the motion trajectories of individual lyric words from correspondence, we analyze the trajectories of the lyric words by k-medoids clustering and dynamic time warping (DTW).

本文言語英語
ホスト出版物のタイトルDocument Analysis Systems - 14th IAPR International Workshop, DAS 2020, Proceedings
編集者Xiang Bai, Dimosthenis Karatzas, Daniel Lopresti
出版社Springer
ページ426-440
ページ数15
ISBN(印刷版)9783030570576
DOI
出版ステータス出版済み - 2020
イベント14th IAPR International Workshop on Document Analysis Systems, DAS 2020 - Wuhan, 中国
継続期間: 7 26 20207 29 2020

出版物シリーズ

名前Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
12116 LNCS
ISSN(印刷版)0302-9743
ISSN(電子版)1611-3349

会議

会議14th IAPR International Workshop on Document Analysis Systems, DAS 2020
Country中国
CityWuhan
Period7/26/207/29/20

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

フィンガープリント 「Lyric video analysis using text detection and tracking」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル