Multi-scale CNN stereo and pattern removal technique for underwater active stereo system

Kazuto Ichimaru, Ryo Furukawa, Hiroshi Kawasaki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Demands on capturing dynamic scenes of underwater environments are rapidly growing. Passive stereo is applicable to capture dynamic scenes, however the shape with textureless surfaces or irregular reflections cannot be recovered by the technique. In our system, we add a pattern projector to the stereo camera pair so that artifi?cial textures are augmented on the objects. To use the system at underwater environments, several problems should be compensated, i.e., refraction, disturbance by fluctuation and bubbles. Further, since surface of the objects are interfered by the bubbles, projected patterns, etc., those noises and patterns should be removed from captured images to recover original texture. To solve these problems, we propose three approaches; a depth-dependent calibration, Convolutional Neural Network(CNN)-stereo method and CNN-based texture recovery method. A depth-dependent calibration I sour analysis to fi?nd the acceptable depth range for approximation by center projection to fi?nd the certain target depth for calibration. In terms of CNN stereo, unlike common CNN based stereo methods which do not consider strong disturbances like refraction or bubbles, we designed a novel CNN architecture for stereo matching using multi-scale information, which is intended to be robust against such disturbances. Finally, we propose a multi-scale method for bubble and a projected-pattern removal method using CNNs to recover original textures. Experimental results are shown to prove the effectiveness of our method compared with the state of the art techniques. Furthermore, reconstruction of a live swimming fi?sh is demonstrated to confi?rm the feasibility of our techniques.

Original languageEnglish
Title of host publicationProceedings - 2018 International Conference on 3D Vision, 3DV 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages61-70
Number of pages10
ISBN (Electronic)9781538684252
DOIs
Publication statusPublished - Oct 12 2018
Event6th International Conference on 3D Vision, 3DV 2018 - Verona, Italy
Duration: Sep 5 2018Sep 8 2018

Publication series

NameProceedings - 2018 International Conference on 3D Vision, 3DV 2018

Other

Other6th International Conference on 3D Vision, 3DV 2018
CountryItaly
CityVerona
Period9/5/189/8/18

Fingerprint

Neural networks
Textures
Calibration
Refraction
Network architecture
Cameras
Recovery

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Cite this

Ichimaru, K., Furukawa, R., & Kawasaki, H. (2018). Multi-scale CNN stereo and pattern removal technique for underwater active stereo system. In Proceedings - 2018 International Conference on 3D Vision, 3DV 2018 (pp. 61-70). [8490956] (Proceedings - 2018 International Conference on 3D Vision, 3DV 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/3DV.2018.00018

Multi-scale CNN stereo and pattern removal technique for underwater active stereo system. / Ichimaru, Kazuto; Furukawa, Ryo; Kawasaki, Hiroshi.

Proceedings - 2018 International Conference on 3D Vision, 3DV 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 61-70 8490956 (Proceedings - 2018 International Conference on 3D Vision, 3DV 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ichimaru, K, Furukawa, R & Kawasaki, H 2018, Multi-scale CNN stereo and pattern removal technique for underwater active stereo system. in Proceedings - 2018 International Conference on 3D Vision, 3DV 2018., 8490956, Proceedings - 2018 International Conference on 3D Vision, 3DV 2018, Institute of Electrical and Electronics Engineers Inc., pp. 61-70, 6th International Conference on 3D Vision, 3DV 2018, Verona, Italy, 9/5/18. https://doi.org/10.1109/3DV.2018.00018
Ichimaru K, Furukawa R, Kawasaki H. Multi-scale CNN stereo and pattern removal technique for underwater active stereo system. In Proceedings - 2018 International Conference on 3D Vision, 3DV 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 61-70. 8490956. (Proceedings - 2018 International Conference on 3D Vision, 3DV 2018). https://doi.org/10.1109/3DV.2018.00018
Ichimaru, Kazuto ; Furukawa, Ryo ; Kawasaki, Hiroshi. / Multi-scale CNN stereo and pattern removal technique for underwater active stereo system. Proceedings - 2018 International Conference on 3D Vision, 3DV 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 61-70 (Proceedings - 2018 International Conference on 3D Vision, 3DV 2018).
@inproceedings{4fcb238adfac4ab589f2f1b369423159,
title = "Multi-scale CNN stereo and pattern removal technique for underwater active stereo system",
abstract = "Demands on capturing dynamic scenes of underwater environments are rapidly growing. Passive stereo is applicable to capture dynamic scenes, however the shape with textureless surfaces or irregular reflections cannot be recovered by the technique. In our system, we add a pattern projector to the stereo camera pair so that artifi?cial textures are augmented on the objects. To use the system at underwater environments, several problems should be compensated, i.e., refraction, disturbance by fluctuation and bubbles. Further, since surface of the objects are interfered by the bubbles, projected patterns, etc., those noises and patterns should be removed from captured images to recover original texture. To solve these problems, we propose three approaches; a depth-dependent calibration, Convolutional Neural Network(CNN)-stereo method and CNN-based texture recovery method. A depth-dependent calibration I sour analysis to fi?nd the acceptable depth range for approximation by center projection to fi?nd the certain target depth for calibration. In terms of CNN stereo, unlike common CNN based stereo methods which do not consider strong disturbances like refraction or bubbles, we designed a novel CNN architecture for stereo matching using multi-scale information, which is intended to be robust against such disturbances. Finally, we propose a multi-scale method for bubble and a projected-pattern removal method using CNNs to recover original textures. Experimental results are shown to prove the effectiveness of our method compared with the state of the art techniques. Furthermore, reconstruction of a live swimming fi?sh is demonstrated to confi?rm the feasibility of our techniques.",
author = "Kazuto Ichimaru and Ryo Furukawa and Hiroshi Kawasaki",
year = "2018",
month = "10",
day = "12",
doi = "10.1109/3DV.2018.00018",
language = "English",
series = "Proceedings - 2018 International Conference on 3D Vision, 3DV 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "61--70",
booktitle = "Proceedings - 2018 International Conference on 3D Vision, 3DV 2018",
address = "United States",

}

TY - GEN

T1 - Multi-scale CNN stereo and pattern removal technique for underwater active stereo system

AU - Ichimaru, Kazuto

AU - Furukawa, Ryo

AU - Kawasaki, Hiroshi

PY - 2018/10/12

Y1 - 2018/10/12

N2 - Demands on capturing dynamic scenes of underwater environments are rapidly growing. Passive stereo is applicable to capture dynamic scenes, however the shape with textureless surfaces or irregular reflections cannot be recovered by the technique. In our system, we add a pattern projector to the stereo camera pair so that artifi?cial textures are augmented on the objects. To use the system at underwater environments, several problems should be compensated, i.e., refraction, disturbance by fluctuation and bubbles. Further, since surface of the objects are interfered by the bubbles, projected patterns, etc., those noises and patterns should be removed from captured images to recover original texture. To solve these problems, we propose three approaches; a depth-dependent calibration, Convolutional Neural Network(CNN)-stereo method and CNN-based texture recovery method. A depth-dependent calibration I sour analysis to fi?nd the acceptable depth range for approximation by center projection to fi?nd the certain target depth for calibration. In terms of CNN stereo, unlike common CNN based stereo methods which do not consider strong disturbances like refraction or bubbles, we designed a novel CNN architecture for stereo matching using multi-scale information, which is intended to be robust against such disturbances. Finally, we propose a multi-scale method for bubble and a projected-pattern removal method using CNNs to recover original textures. Experimental results are shown to prove the effectiveness of our method compared with the state of the art techniques. Furthermore, reconstruction of a live swimming fi?sh is demonstrated to confi?rm the feasibility of our techniques.

AB - Demands on capturing dynamic scenes of underwater environments are rapidly growing. Passive stereo is applicable to capture dynamic scenes, however the shape with textureless surfaces or irregular reflections cannot be recovered by the technique. In our system, we add a pattern projector to the stereo camera pair so that artifi?cial textures are augmented on the objects. To use the system at underwater environments, several problems should be compensated, i.e., refraction, disturbance by fluctuation and bubbles. Further, since surface of the objects are interfered by the bubbles, projected patterns, etc., those noises and patterns should be removed from captured images to recover original texture. To solve these problems, we propose three approaches; a depth-dependent calibration, Convolutional Neural Network(CNN)-stereo method and CNN-based texture recovery method. A depth-dependent calibration I sour analysis to fi?nd the acceptable depth range for approximation by center projection to fi?nd the certain target depth for calibration. In terms of CNN stereo, unlike common CNN based stereo methods which do not consider strong disturbances like refraction or bubbles, we designed a novel CNN architecture for stereo matching using multi-scale information, which is intended to be robust against such disturbances. Finally, we propose a multi-scale method for bubble and a projected-pattern removal method using CNNs to recover original textures. Experimental results are shown to prove the effectiveness of our method compared with the state of the art techniques. Furthermore, reconstruction of a live swimming fi?sh is demonstrated to confi?rm the feasibility of our techniques.

UR - http://www.scopus.com/inward/record.url?scp=85056753958&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056753958&partnerID=8YFLogxK

U2 - 10.1109/3DV.2018.00018

DO - 10.1109/3DV.2018.00018

M3 - Conference contribution

T3 - Proceedings - 2018 International Conference on 3D Vision, 3DV 2018

SP - 61

EP - 70

BT - Proceedings - 2018 International Conference on 3D Vision, 3DV 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -