Detecting anomalous regions from an image based on deep captioning

Yusuke Hatae, Qingpu Yang, Muhammad Fikko Fadjrimiratno, Yuanyuan Li, Tetsu Matsukawa, Einoshin Suzuki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper we propose a one-class anomalous region detection method from an image based on deep captioning. Such a method can be installed on an autonomous mobile robot, which reports anomalies from observation without any human supervision and would interest a wide range of researchers, practitioners, and users. In addition to image features, which were used by conventional methods, our method exploits recent advances in deep captioning, which is based on deep neural networks trained on a large-scale data on image - caption pairs, enabling anomaly detection in the semantic level. Incremental clustering is adopted so that the robot is able to model its observation into a set of clusters and report substantially new observations as anomalies. Extensive experiments using two real-world data demonstrate the superiority of our method in terms of recall, precision, F measure, and AUC over the traditional approach. The experiments also show that our method exhibits excellent learning curve and low threshold dependency.

Original languageEnglish
Title of host publicationVISAPP
EditorsGiovanni Maria Farinella, Petia Radeva, Jose Braz
PublisherSciTePress
Pages326-335
Number of pages10
ISBN (Electronic)9789897584022
Publication statusPublished - Jan 1 2020
Event15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISIGRAPP 2020 - Valletta, Malta
Duration: Feb 27 2020Feb 29 2020

Publication series

NameVISIGRAPP 2020 - Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications
Volume5

Conference

Conference15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISIGRAPP 2020
CountryMalta
CityValletta
Period2/27/202/29/20

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design
  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of 'Detecting anomalous regions from an image based on deep captioning'. Together they form a unique fingerprint.

  • Cite this

    Hatae, Y., Yang, Q., Fadjrimiratno, M. F., Li, Y., Matsukawa, T., & Suzuki, E. (2020). Detecting anomalous regions from an image based on deep captioning. In G. M. Farinella, P. Radeva, & J. Braz (Eds.), VISAPP (pp. 326-335). (VISIGRAPP 2020 - Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications; Vol. 5). SciTePress.