Temporal Shape Super-Resolution by Intra-frame Motion Encoding Using High-fps Structured Light

Yuki Shiba, Satoshi Ono, Ryo Furukawa, Shinsaku Hiura, Hiroshi Kawasaki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

One of the solutions of depth imaging of moving scene is to project a static pattern on the object and use just a single image for reconstruction. However, if the motion of the object is too fast with respect to the exposure time of the image sensor, patterns on the captured image are blurred and reconstruction fails. In this paper, we impose multiple projection patterns into each single captured image to realize temporal super resolution of the depth image sequences. With our method, multiple patterns are projected onto the object with higher fps than possible with a camera. In this case, the observed pattern varies depending on the depth and motion of the object, so we can extract temporal information of the scene from each single image. The decoding process is realized using a learning-based approach where no geometric calibration is needed. Experiments confirm the effectiveness of our method where sequential shapes are reconstructed from a single image. Both quantitative evaluations and comparisons with recent techniques were also conducted.

Original languageEnglish
Title of host publicationProceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages115-123
Number of pages9
ISBN (Electronic)9781538610329
DOIs
Publication statusPublished - Dec 22 2017
Event16th IEEE International Conference on Computer Vision, ICCV 2017 - Venice, Italy
Duration: Oct 22 2017Oct 29 2017

Publication series

NameProceedings of the IEEE International Conference on Computer Vision
Volume2017-October
ISSN (Print)1550-5499

Other

Other16th IEEE International Conference on Computer Vision, ICCV 2017
CountryItaly
CityVenice
Period10/22/1710/29/17

Fingerprint

Image sensors
Decoding
Cameras
Calibration
Imaging techniques
Experiments

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Shiba, Y., Ono, S., Furukawa, R., Hiura, S., & Kawasaki, H. (2017). Temporal Shape Super-Resolution by Intra-frame Motion Encoding Using High-fps Structured Light. In Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017 (pp. 115-123). [8237284] (Proceedings of the IEEE International Conference on Computer Vision; Vol. 2017-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCV.2017.22

Temporal Shape Super-Resolution by Intra-frame Motion Encoding Using High-fps Structured Light. / Shiba, Yuki; Ono, Satoshi; Furukawa, Ryo; Hiura, Shinsaku; Kawasaki, Hiroshi.

Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 115-123 8237284 (Proceedings of the IEEE International Conference on Computer Vision; Vol. 2017-October).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Shiba, Y, Ono, S, Furukawa, R, Hiura, S & Kawasaki, H 2017, Temporal Shape Super-Resolution by Intra-frame Motion Encoding Using High-fps Structured Light. in Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017., 8237284, Proceedings of the IEEE International Conference on Computer Vision, vol. 2017-October, Institute of Electrical and Electronics Engineers Inc., pp. 115-123, 16th IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, 10/22/17. https://doi.org/10.1109/ICCV.2017.22
Shiba Y, Ono S, Furukawa R, Hiura S, Kawasaki H. Temporal Shape Super-Resolution by Intra-frame Motion Encoding Using High-fps Structured Light. In Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 115-123. 8237284. (Proceedings of the IEEE International Conference on Computer Vision). https://doi.org/10.1109/ICCV.2017.22
Shiba, Yuki ; Ono, Satoshi ; Furukawa, Ryo ; Hiura, Shinsaku ; Kawasaki, Hiroshi. / Temporal Shape Super-Resolution by Intra-frame Motion Encoding Using High-fps Structured Light. Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 115-123 (Proceedings of the IEEE International Conference on Computer Vision).
@inproceedings{b0588d49d3b94a41b18ff21538352a01,
title = "Temporal Shape Super-Resolution by Intra-frame Motion Encoding Using High-fps Structured Light",
abstract = "One of the solutions of depth imaging of moving scene is to project a static pattern on the object and use just a single image for reconstruction. However, if the motion of the object is too fast with respect to the exposure time of the image sensor, patterns on the captured image are blurred and reconstruction fails. In this paper, we impose multiple projection patterns into each single captured image to realize temporal super resolution of the depth image sequences. With our method, multiple patterns are projected onto the object with higher fps than possible with a camera. In this case, the observed pattern varies depending on the depth and motion of the object, so we can extract temporal information of the scene from each single image. The decoding process is realized using a learning-based approach where no geometric calibration is needed. Experiments confirm the effectiveness of our method where sequential shapes are reconstructed from a single image. Both quantitative evaluations and comparisons with recent techniques were also conducted.",
author = "Yuki Shiba and Satoshi Ono and Ryo Furukawa and Shinsaku Hiura and Hiroshi Kawasaki",
year = "2017",
month = "12",
day = "22",
doi = "10.1109/ICCV.2017.22",
language = "English",
series = "Proceedings of the IEEE International Conference on Computer Vision",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "115--123",
booktitle = "Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017",
address = "United States",

}

TY - GEN

T1 - Temporal Shape Super-Resolution by Intra-frame Motion Encoding Using High-fps Structured Light

AU - Shiba, Yuki

AU - Ono, Satoshi

AU - Furukawa, Ryo

AU - Hiura, Shinsaku

AU - Kawasaki, Hiroshi

PY - 2017/12/22

Y1 - 2017/12/22

N2 - One of the solutions of depth imaging of moving scene is to project a static pattern on the object and use just a single image for reconstruction. However, if the motion of the object is too fast with respect to the exposure time of the image sensor, patterns on the captured image are blurred and reconstruction fails. In this paper, we impose multiple projection patterns into each single captured image to realize temporal super resolution of the depth image sequences. With our method, multiple patterns are projected onto the object with higher fps than possible with a camera. In this case, the observed pattern varies depending on the depth and motion of the object, so we can extract temporal information of the scene from each single image. The decoding process is realized using a learning-based approach where no geometric calibration is needed. Experiments confirm the effectiveness of our method where sequential shapes are reconstructed from a single image. Both quantitative evaluations and comparisons with recent techniques were also conducted.

AB - One of the solutions of depth imaging of moving scene is to project a static pattern on the object and use just a single image for reconstruction. However, if the motion of the object is too fast with respect to the exposure time of the image sensor, patterns on the captured image are blurred and reconstruction fails. In this paper, we impose multiple projection patterns into each single captured image to realize temporal super resolution of the depth image sequences. With our method, multiple patterns are projected onto the object with higher fps than possible with a camera. In this case, the observed pattern varies depending on the depth and motion of the object, so we can extract temporal information of the scene from each single image. The decoding process is realized using a learning-based approach where no geometric calibration is needed. Experiments confirm the effectiveness of our method where sequential shapes are reconstructed from a single image. Both quantitative evaluations and comparisons with recent techniques were also conducted.

UR - http://www.scopus.com/inward/record.url?scp=85041921526&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85041921526&partnerID=8YFLogxK

U2 - 10.1109/ICCV.2017.22

DO - 10.1109/ICCV.2017.22

M3 - Conference contribution

AN - SCOPUS:85041921526

T3 - Proceedings of the IEEE International Conference on Computer Vision

SP - 115

EP - 123

BT - Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -