Learning-based feature extraction for active 3D scan with reducing color crosstalk of multiple pattern projections

Ryusuke Sagawa, Ryo Furukawa, Akiko Matsumoto, Hiroshi Kawasaki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

3D reconstruction methods based on active stereo technique have been widely used for many practical systems. Many of these systems are configured with a single camera and a single projector. Since such systems can only capture one side of the target object, several attempts have been conducted to enlarge the captured area, especially multi-projector systems attract many researchers. For multi-projector based systems, overlap between multiple pattern projections is a serious problem. Even if different color channels are used for each projector, complete separation is not possible because of color crosstalks. Another open problem is decoding errors of the projected patterns, which causes a failure on extracting positional information of the projected pattern form the captured image. Among several reasons for such errors, color crosstalks are crucial because their features are similar to the main signal and difficult to be decomposed. In this paper, we solve these problems by utilizing machine learning techniques where a convolutional neural network is trained to extract low dimensional pattern features for each projector. In addition, it is trained to suppress the color crosstalks from different projectors. Using this new technique, we succeeded in reconstructing 3D shapes from images where multiple patterns are overlapped.

Original languageEnglish
Title of host publicationICRA 2017 - IEEE International Conference on Robotics and Automation
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages5074-5080
Number of pages7
ISBN (Electronic)9781509046331
DOIs
Publication statusPublished - Jul 21 2017
Externally publishedYes
Event2017 IEEE International Conference on Robotics and Automation, ICRA 2017 - Singapore, Singapore
Duration: May 29 2017Jun 3 2017

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
ISSN (Print)1050-4729

Other

Other2017 IEEE International Conference on Robotics and Automation, ICRA 2017
CountrySingapore
CitySingapore
Period5/29/176/3/17

Fingerprint

Crosstalk
Feature extraction
Color
Decoding
Learning systems
Cameras
Neural networks

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Systems Engineering
  • Artificial Intelligence
  • Electrical and Electronic Engineering

Cite this

Sagawa, R., Furukawa, R., Matsumoto, A., & Kawasaki, H. (2017). Learning-based feature extraction for active 3D scan with reducing color crosstalk of multiple pattern projections. In ICRA 2017 - IEEE International Conference on Robotics and Automation (pp. 5074-5080). [7989592] (Proceedings - IEEE International Conference on Robotics and Automation). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICRA.2017.7989592

Learning-based feature extraction for active 3D scan with reducing color crosstalk of multiple pattern projections. / Sagawa, Ryusuke; Furukawa, Ryo; Matsumoto, Akiko; Kawasaki, Hiroshi.

ICRA 2017 - IEEE International Conference on Robotics and Automation. Institute of Electrical and Electronics Engineers Inc., 2017. p. 5074-5080 7989592 (Proceedings - IEEE International Conference on Robotics and Automation).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sagawa, R, Furukawa, R, Matsumoto, A & Kawasaki, H 2017, Learning-based feature extraction for active 3D scan with reducing color crosstalk of multiple pattern projections. in ICRA 2017 - IEEE International Conference on Robotics and Automation., 7989592, Proceedings - IEEE International Conference on Robotics and Automation, Institute of Electrical and Electronics Engineers Inc., pp. 5074-5080, 2017 IEEE International Conference on Robotics and Automation, ICRA 2017, Singapore, Singapore, 5/29/17. https://doi.org/10.1109/ICRA.2017.7989592
Sagawa R, Furukawa R, Matsumoto A, Kawasaki H. Learning-based feature extraction for active 3D scan with reducing color crosstalk of multiple pattern projections. In ICRA 2017 - IEEE International Conference on Robotics and Automation. Institute of Electrical and Electronics Engineers Inc. 2017. p. 5074-5080. 7989592. (Proceedings - IEEE International Conference on Robotics and Automation). https://doi.org/10.1109/ICRA.2017.7989592
Sagawa, Ryusuke ; Furukawa, Ryo ; Matsumoto, Akiko ; Kawasaki, Hiroshi. / Learning-based feature extraction for active 3D scan with reducing color crosstalk of multiple pattern projections. ICRA 2017 - IEEE International Conference on Robotics and Automation. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 5074-5080 (Proceedings - IEEE International Conference on Robotics and Automation).
@inproceedings{7d7a7cf14e2b4e338e0c439cb099df0c,
title = "Learning-based feature extraction for active 3D scan with reducing color crosstalk of multiple pattern projections",
abstract = "3D reconstruction methods based on active stereo technique have been widely used for many practical systems. Many of these systems are configured with a single camera and a single projector. Since such systems can only capture one side of the target object, several attempts have been conducted to enlarge the captured area, especially multi-projector systems attract many researchers. For multi-projector based systems, overlap between multiple pattern projections is a serious problem. Even if different color channels are used for each projector, complete separation is not possible because of color crosstalks. Another open problem is decoding errors of the projected patterns, which causes a failure on extracting positional information of the projected pattern form the captured image. Among several reasons for such errors, color crosstalks are crucial because their features are similar to the main signal and difficult to be decomposed. In this paper, we solve these problems by utilizing machine learning techniques where a convolutional neural network is trained to extract low dimensional pattern features for each projector. In addition, it is trained to suppress the color crosstalks from different projectors. Using this new technique, we succeeded in reconstructing 3D shapes from images where multiple patterns are overlapped.",
author = "Ryusuke Sagawa and Ryo Furukawa and Akiko Matsumoto and Hiroshi Kawasaki",
year = "2017",
month = "7",
day = "21",
doi = "10.1109/ICRA.2017.7989592",
language = "English",
series = "Proceedings - IEEE International Conference on Robotics and Automation",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "5074--5080",
booktitle = "ICRA 2017 - IEEE International Conference on Robotics and Automation",
address = "United States",

}

TY - GEN

T1 - Learning-based feature extraction for active 3D scan with reducing color crosstalk of multiple pattern projections

AU - Sagawa, Ryusuke

AU - Furukawa, Ryo

AU - Matsumoto, Akiko

AU - Kawasaki, Hiroshi

PY - 2017/7/21

Y1 - 2017/7/21

N2 - 3D reconstruction methods based on active stereo technique have been widely used for many practical systems. Many of these systems are configured with a single camera and a single projector. Since such systems can only capture one side of the target object, several attempts have been conducted to enlarge the captured area, especially multi-projector systems attract many researchers. For multi-projector based systems, overlap between multiple pattern projections is a serious problem. Even if different color channels are used for each projector, complete separation is not possible because of color crosstalks. Another open problem is decoding errors of the projected patterns, which causes a failure on extracting positional information of the projected pattern form the captured image. Among several reasons for such errors, color crosstalks are crucial because their features are similar to the main signal and difficult to be decomposed. In this paper, we solve these problems by utilizing machine learning techniques where a convolutional neural network is trained to extract low dimensional pattern features for each projector. In addition, it is trained to suppress the color crosstalks from different projectors. Using this new technique, we succeeded in reconstructing 3D shapes from images where multiple patterns are overlapped.

AB - 3D reconstruction methods based on active stereo technique have been widely used for many practical systems. Many of these systems are configured with a single camera and a single projector. Since such systems can only capture one side of the target object, several attempts have been conducted to enlarge the captured area, especially multi-projector systems attract many researchers. For multi-projector based systems, overlap between multiple pattern projections is a serious problem. Even if different color channels are used for each projector, complete separation is not possible because of color crosstalks. Another open problem is decoding errors of the projected patterns, which causes a failure on extracting positional information of the projected pattern form the captured image. Among several reasons for such errors, color crosstalks are crucial because their features are similar to the main signal and difficult to be decomposed. In this paper, we solve these problems by utilizing machine learning techniques where a convolutional neural network is trained to extract low dimensional pattern features for each projector. In addition, it is trained to suppress the color crosstalks from different projectors. Using this new technique, we succeeded in reconstructing 3D shapes from images where multiple patterns are overlapped.

UR - http://www.scopus.com/inward/record.url?scp=85028021187&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85028021187&partnerID=8YFLogxK

U2 - 10.1109/ICRA.2017.7989592

DO - 10.1109/ICRA.2017.7989592

M3 - Conference contribution

T3 - Proceedings - IEEE International Conference on Robotics and Automation

SP - 5074

EP - 5080

BT - ICRA 2017 - IEEE International Conference on Robotics and Automation

PB - Institute of Electrical and Electronics Engineers Inc.

ER -