3D endoscope system with AR display superimposing dense and wide-angle-of-view 3D points obtained by using micro pattern projector

Michihiro Mikamo, Ryo Furukawa, Shiro Oka, Takahiro Kotachi, Yuki Okamoto, Shinji Tanaka, Ryusuke Sagawa, Hiroshi Kawasaki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In recent years, augmented reality (AR) technologies have been widespread for supporting various kinds of tasks, by superimposing useful information on the users' view of the real environments. In endoscopic diagnosis, AR systems can be helpful as an aid in presenting information to endoscopists who have their hands full. In this paper, we propose a system that can superimpose shapes, which are reconstructed from an endoscope image, onto the field of view. The feature of the proposed system is that it reconstructs 3D shapes from the images captured by the endoscope and superimposes them onto the real views. As a result, the superimposed view allows the doctor to keep operating the endoscope while observing the patient's internal body with additional information. The proposed system is composed of the reconstruction module and the display module. The reconstruction module is for acquiring 3D shapes based on an active stereo method. In particular, we propose a novel projection pattern that can reconstruct wide areas of the endoscopic view. The display module shows the 3D shape obtained by the reconstructed module, superimposing on the field of view. In the experiments, we show that it is possible to perform a wide range of dense 3D reconstructions using the new projection patterns. In addition, we confirmed the usefulness of the AR system by interviewing medical doctors.

Original languageEnglish
Title of host publication44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages881-885
Number of pages5
ISBN (Electronic)9781728127828
DOIs
Publication statusPublished - 2022
Event44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022 - Glasgow, United Kingdom
Duration: Jul 11 2022Jul 15 2022

Publication series

NameProceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
Volume2022-July
ISSN (Print)1557-170X

Conference

Conference44th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2022
Country/TerritoryUnited Kingdom
CityGlasgow
Period7/11/227/15/22

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Biomedical Engineering
  • Computer Vision and Pattern Recognition
  • Health Informatics

Fingerprint

Dive into the research topics of '3D endoscope system with AR display superimposing dense and wide-angle-of-view 3D points obtained by using micro pattern projector'. Together they form a unique fingerprint.

Cite this