Wide-area shape reconstruction by 3D endoscopic system based on CNN decoding, shape registration and fusion

Ryo Furukawa, Masaki Mizomori, Shinsaku Hiura, Shiro Oka, Shinji Tanaka, Hiroshi Kawasaki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

For effective in situ endoscopic diagnosis and treatment, dense and large areal shape reconstruction is important. For this purpose, we develop 3D endoscopic systems based on active stereo, which projects a grid pattern where grid points are coded by line gaps. One problem of the previous works was that success or failure of 3D reconstruction depends on the stability of feature extraction from the images captured by the endoscope camera. Subsurface scattering or specularities on bio-tissues make this problem difficult. Another problem was that shape reconstruction area was relatively small because of limited field of view of the pattern projector compared to that of the camera. In this paper, to solve the first problem, learning-based approach, i.e., U-Nets, for efficient detection of grid lines and codes at the detected grid points under severe conditions, is proposed. To solve the second problem, an online shape-registration and merging algorithm for sequential frames is proposed. In the experiments, we have shown that we can train U-Nets to extract those features effectively for three specimens of cancers, and also conducted 3D scanning of shapes of a stomach phantom model and a surface inside a human mouth, in which wide-area surfaces are successfully recovered by shape registration and merging.

Original languageEnglish
Title of host publicationOR 2.0 Context-Aware Operating Theaters, Computer Assisted Robotic Endoscopy, Clinical Image-Based Procedures, and Skin Image Analysis - 1st International Workshop, OR 2.0 2018 5th International Workshop, CARE 2018, 7th International Workshop, CLIP 2018, 3rd International Workshop, ISIC 2018 Held in Conjunction with MICCAI 2018
EditorsAnand Malpani, Marco A. Zenati, Cristina Oyarzun Laura, M. Emre Celebi, Duygu Sarikaya, Noel C. Codella, Allan Halpern, Marius Erdt, Lena Maier-Hein, Luo Xiongbiao, Stefan Wesarg, Danail Stoyanov, Zeike Taylor, Klaus Drechsler, Kristin Dana, Anne Martel, Raj Shekhar, Sandrine De Ribaupierre, Tobias Reichl, Jonathan McLeod, Miguel Angel González Ballester, Toby Collins, Marius George Linguraru
PublisherSpringer Verlag
Pages139-150
Number of pages12
ISBN (Print)9783030012007
DOIs
Publication statusPublished - Jan 1 2018
Event1st International Workshop on OR 2.0 Context-Aware Operating Theaters, OR 2.0 2018, 5th International Workshop on Computer Assisted Robotic Endoscopy, CARE 2018, 7th International Workshop on Clinical Image-Based Procedures, CLIP 2018, and 1st International Workshop on Skin Image Analysis, ISIC 2018, held in conjunction with the 21st International Conference on Medical Imaging and Computer-Assisted Intervention, MICCAI 2018 - Granada, Spain
Duration: Sep 16 2018Sep 20 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11041 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other1st International Workshop on OR 2.0 Context-Aware Operating Theaters, OR 2.0 2018, 5th International Workshop on Computer Assisted Robotic Endoscopy, CARE 2018, 7th International Workshop on Clinical Image-Based Procedures, CLIP 2018, and 1st International Workshop on Skin Image Analysis, ISIC 2018, held in conjunction with the 21st International Conference on Medical Imaging and Computer-Assisted Intervention, MICCAI 2018
CountrySpain
CityGranada
Period9/16/189/20/18

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Wide-area shape reconstruction by 3D endoscopic system based on CNN decoding, shape registration and fusion'. Together they form a unique fingerprint.

  • Cite this

    Furukawa, R., Mizomori, M., Hiura, S., Oka, S., Tanaka, S., & Kawasaki, H. (2018). Wide-area shape reconstruction by 3D endoscopic system based on CNN decoding, shape registration and fusion. In A. Malpani, M. A. Zenati, C. Oyarzun Laura, M. E. Celebi, D. Sarikaya, N. C. Codella, A. Halpern, M. Erdt, L. Maier-Hein, L. Xiongbiao, S. Wesarg, D. Stoyanov, Z. Taylor, K. Drechsler, K. Dana, A. Martel, R. Shekhar, S. De Ribaupierre, T. Reichl, J. McLeod, M. A. González Ballester, T. Collins, ... M. G. Linguraru (Eds.), OR 2.0 Context-Aware Operating Theaters, Computer Assisted Robotic Endoscopy, Clinical Image-Based Procedures, and Skin Image Analysis - 1st International Workshop, OR 2.0 2018 5th International Workshop, CARE 2018, 7th International Workshop, CLIP 2018, 3rd International Workshop, ISIC 2018 Held in Conjunction with MICCAI 2018 (pp. 139-150). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11041 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-01201-4_16