CNN based dense underwater 3D scene reconstruction by transfer learning using bubble database

Kazuto Ichimaru, Ryo Furukawa, Hiroshi Kawasaki

研究成果: 著書/レポートタイプへの貢献会議での発言

抄録

Dense 3D shape acquisition of swimming human or live fish is an important research topic for sports, biological science and so on. For this purpose, active stereo sensor is usually used in the air, however it cannot be applied to the underwater environment because of refraction, strong light attenuation and severe interference of bubbles. Passive stereo is a simple solution for capturing dynamic scenes at underwater environment, however the shape with textureless surfaces or irregular reflections cannot be recovered. Recently, the stereo camera pair with a pattern projector for adding artificial textures on the objects is proposed. However, to use the system for underwater environment, several problems should be compensated, i.e., disturbance by fluctuation and bubbles. Simple solution is to use convolutional neural network for stereo to cancel the effects of bubbles and/or water fluctuation. Since it is not easy to train CNN with small size of database with large variation, we develop a special bubble generation device to efficiently create real bubble database of multiple size and density. In addition, we propose a transfer learning technique for multi-scale CNN to effectively remove bubbles and projected-patterns on the object. Further, we develop a real system and actually captured live swimming human, which has not been done before. Experiments are conducted to show the effectiveness of our method compared with the state of the art techniques.

元の言語英語
ホスト出版物のタイトルProceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019
出版者Institute of Electrical and Electronics Engineers Inc.
ページ1543-1552
ページ数10
ISBN(電子版)9781728119755
DOI
出版物ステータス出版済み - 3 4 2019
イベント19th IEEE Winter Conference on Applications of Computer Vision, WACV 2019 - Waikoloa Village, 米国
継続期間: 1 7 20191 11 2019

出版物シリーズ

名前Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019

会議

会議19th IEEE Winter Conference on Applications of Computer Vision, WACV 2019
米国
Waikoloa Village
期間1/7/191/11/19

Fingerprint

Light refraction
Sports
Fish
Textures
Cameras
Neural networks
Sensors
Air
Water
Experiments
Swimming

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition
  • Computer Science Applications

これを引用

Ichimaru, K., Furukawa, R., & Kawasaki, H. (2019). CNN based dense underwater 3D scene reconstruction by transfer learning using bubble database. : Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019 (pp. 1543-1552). [8659224] (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WACV.2019.00169

CNN based dense underwater 3D scene reconstruction by transfer learning using bubble database. / Ichimaru, Kazuto; Furukawa, Ryo; Kawasaki, Hiroshi.

Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 1543-1552 8659224 (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019).

研究成果: 著書/レポートタイプへの貢献会議での発言

Ichimaru, K, Furukawa, R & Kawasaki, H 2019, CNN based dense underwater 3D scene reconstruction by transfer learning using bubble database. : Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019., 8659224, Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019, Institute of Electrical and Electronics Engineers Inc., pp. 1543-1552, 19th IEEE Winter Conference on Applications of Computer Vision, WACV 2019, Waikoloa Village, 米国, 1/7/19. https://doi.org/10.1109/WACV.2019.00169
Ichimaru K, Furukawa R, Kawasaki H. CNN based dense underwater 3D scene reconstruction by transfer learning using bubble database. : Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 1543-1552. 8659224. (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019). https://doi.org/10.1109/WACV.2019.00169
Ichimaru, Kazuto ; Furukawa, Ryo ; Kawasaki, Hiroshi. / CNN based dense underwater 3D scene reconstruction by transfer learning using bubble database. Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 1543-1552 (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019).
@inproceedings{e83e58aa8a83459ea97ee2e98a2c3ecc,
title = "CNN based dense underwater 3D scene reconstruction by transfer learning using bubble database",
abstract = "Dense 3D shape acquisition of swimming human or live fish is an important research topic for sports, biological science and so on. For this purpose, active stereo sensor is usually used in the air, however it cannot be applied to the underwater environment because of refraction, strong light attenuation and severe interference of bubbles. Passive stereo is a simple solution for capturing dynamic scenes at underwater environment, however the shape with textureless surfaces or irregular reflections cannot be recovered. Recently, the stereo camera pair with a pattern projector for adding artificial textures on the objects is proposed. However, to use the system for underwater environment, several problems should be compensated, i.e., disturbance by fluctuation and bubbles. Simple solution is to use convolutional neural network for stereo to cancel the effects of bubbles and/or water fluctuation. Since it is not easy to train CNN with small size of database with large variation, we develop a special bubble generation device to efficiently create real bubble database of multiple size and density. In addition, we propose a transfer learning technique for multi-scale CNN to effectively remove bubbles and projected-patterns on the object. Further, we develop a real system and actually captured live swimming human, which has not been done before. Experiments are conducted to show the effectiveness of our method compared with the state of the art techniques.",
author = "Kazuto Ichimaru and Ryo Furukawa and Hiroshi Kawasaki",
year = "2019",
month = "3",
day = "4",
doi = "10.1109/WACV.2019.00169",
language = "English",
series = "Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "1543--1552",
booktitle = "Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019",
address = "United States",

}

TY - GEN

T1 - CNN based dense underwater 3D scene reconstruction by transfer learning using bubble database

AU - Ichimaru, Kazuto

AU - Furukawa, Ryo

AU - Kawasaki, Hiroshi

PY - 2019/3/4

Y1 - 2019/3/4

N2 - Dense 3D shape acquisition of swimming human or live fish is an important research topic for sports, biological science and so on. For this purpose, active stereo sensor is usually used in the air, however it cannot be applied to the underwater environment because of refraction, strong light attenuation and severe interference of bubbles. Passive stereo is a simple solution for capturing dynamic scenes at underwater environment, however the shape with textureless surfaces or irregular reflections cannot be recovered. Recently, the stereo camera pair with a pattern projector for adding artificial textures on the objects is proposed. However, to use the system for underwater environment, several problems should be compensated, i.e., disturbance by fluctuation and bubbles. Simple solution is to use convolutional neural network for stereo to cancel the effects of bubbles and/or water fluctuation. Since it is not easy to train CNN with small size of database with large variation, we develop a special bubble generation device to efficiently create real bubble database of multiple size and density. In addition, we propose a transfer learning technique for multi-scale CNN to effectively remove bubbles and projected-patterns on the object. Further, we develop a real system and actually captured live swimming human, which has not been done before. Experiments are conducted to show the effectiveness of our method compared with the state of the art techniques.

AB - Dense 3D shape acquisition of swimming human or live fish is an important research topic for sports, biological science and so on. For this purpose, active stereo sensor is usually used in the air, however it cannot be applied to the underwater environment because of refraction, strong light attenuation and severe interference of bubbles. Passive stereo is a simple solution for capturing dynamic scenes at underwater environment, however the shape with textureless surfaces or irregular reflections cannot be recovered. Recently, the stereo camera pair with a pattern projector for adding artificial textures on the objects is proposed. However, to use the system for underwater environment, several problems should be compensated, i.e., disturbance by fluctuation and bubbles. Simple solution is to use convolutional neural network for stereo to cancel the effects of bubbles and/or water fluctuation. Since it is not easy to train CNN with small size of database with large variation, we develop a special bubble generation device to efficiently create real bubble database of multiple size and density. In addition, we propose a transfer learning technique for multi-scale CNN to effectively remove bubbles and projected-patterns on the object. Further, we develop a real system and actually captured live swimming human, which has not been done before. Experiments are conducted to show the effectiveness of our method compared with the state of the art techniques.

UR - http://www.scopus.com/inward/record.url?scp=85063586273&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063586273&partnerID=8YFLogxK

U2 - 10.1109/WACV.2019.00169

DO - 10.1109/WACV.2019.00169

M3 - Conference contribution

T3 - Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019

SP - 1543

EP - 1552

BT - Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -