TU-Net and TDeepLab: Deep Learning-Based Terrain Classification Robust to Illumination Changes, Combining Visible and Thermal Imagery

Yumi Iwashita, Kazuto Nakashima, Adrian Stoica, Ryo Kurazume

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

In this paper we propose two novel deep learning-based terrain classification methods robust to illumination changes. The use of cameras is challenged by a variety of factors, of most importance being the changes in illumination. On the other hand, since the temperature of various types of terrains depends on the thermal characteristics of the terrain, the terrain classification can be aided by utilizing the thermal information in addition to visible information. Thus we propose 'TU-Net (Two U-Net)' based on the U-Net and 'TDeepLab (Two DeepLab)' based on DeepLab, which combine visible and thermal images and train the network robust to illumination changes implicitly. To improve the network's learning capability, we expand the proposed methods to the Siamese-based method, which explicitly trains the network to be robust to illumination changes. We also investigate multiple options to fuse the visible and thermal images at at the bottom layer, middle layer, or the top layer of the network. We evaluate the proposed methods with a challenging new dataset consisting of visible and thermal images, which were collected from 10 am till 5 pm (after sunset), and we show the effectiveness of the proposed methods.

Original languageEnglish
Title of host publicationProceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages280-285
Number of pages6
ISBN (Electronic)9781728111988
DOIs
Publication statusPublished - Apr 22 2019
Event2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 - San Jose, United States
Duration: Mar 28 2019Mar 30 2019

Publication series

NameProceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019

Conference

Conference2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
CountryUnited States
CitySan Jose
Period3/28/193/30/19

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Signal Processing
  • Media Technology

Fingerprint Dive into the research topics of 'TU-Net and TDeepLab: Deep Learning-Based Terrain Classification Robust to Illumination Changes, Combining Visible and Thermal Imagery'. Together they form a unique fingerprint.

  • Cite this

    Iwashita, Y., Nakashima, K., Stoica, A., & Kurazume, R. (2019). TU-Net and TDeepLab: Deep Learning-Based Terrain Classification Robust to Illumination Changes, Combining Visible and Thermal Imagery. In Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019 (pp. 280-285). [8695421] (Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/MIPR.2019.00057