High-speed imaging using CMOS image sensor with quasi pixel-wise exposure

Hajime Nagahara, Toshiki Sonoda, Kenta Endo, Yukinobu Sugiyama, Rin Ichiro Taniguchi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)

Abstract

Several recent studies in compressive video sensing have realized scene capture beyond the fundamental trade-off limit between spatial resolution and temporal resolution using random space-time sampling. However, most of these studies showed results for higher frame rate video that were produced by simulation experiments or using an optically simulated random sampling camera, because there are currently no commercially available image sensors with random exposure or sampling capabilities. We fabricated a prototype complementary metal oxide semiconductor (CMOS) image sensor with quasi pixel-wise exposure timing that can realize nonuniform space-time sampling. The prototype sensor can reset exposures independently by columns and fix these amount of exposure by rows for each 8×8 pixel block. This CMOS sensor is not fully controllable via the pixels, and has line-dependent controls, but it offers flexibility when compared with regular CMOS or charge-coupled device sensors with global or rolling shutters. We propose a method to realize pseudo-random sampling for high-speed video acquisition that uses the flexibility of the CMOS sensor. We reconstruct the high-speed video sequence from the images produced by pseudo-random sampling using an over-complete dictionary. The proposed method also removes the rolling shutter effect from the reconstructed video.

Original languageEnglish
Title of host publication2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781467386234
DOIs
Publication statusPublished - Jun 15 2016
Event2016 IEEE International Conference on Computational Photography, ICCP 2016 - Evanston, United States
Duration: May 13 2016May 15 2016

Publication series

Name2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings

Other

Other2016 IEEE International Conference on Computational Photography, ICCP 2016
CountryUnited States
CityEvanston
Period5/13/165/15/16

Fingerprint

Image sensors
CMOS
Pixels
pixels
high speed
Sampling
random sampling
Imaging techniques
sensors
Metals
shutters
sampling
Sensors
flexibility
prototypes
dictionaries
Glossaries
temporal resolution
Charge coupled devices
fixing

All Science Journal Classification (ASJC) codes

  • Media Technology
  • Instrumentation

Cite this

Nagahara, H., Sonoda, T., Endo, K., Sugiyama, Y., & Taniguchi, R. I. (2016). High-speed imaging using CMOS image sensor with quasi pixel-wise exposure. In 2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings [7492875] (2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCPHOT.2016.7492875

High-speed imaging using CMOS image sensor with quasi pixel-wise exposure. / Nagahara, Hajime; Sonoda, Toshiki; Endo, Kenta; Sugiyama, Yukinobu; Taniguchi, Rin Ichiro.

2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2016. 7492875 (2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Nagahara, H, Sonoda, T, Endo, K, Sugiyama, Y & Taniguchi, RI 2016, High-speed imaging using CMOS image sensor with quasi pixel-wise exposure. in 2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings., 7492875, 2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings, Institute of Electrical and Electronics Engineers Inc., 2016 IEEE International Conference on Computational Photography, ICCP 2016, Evanston, United States, 5/13/16. https://doi.org/10.1109/ICCPHOT.2016.7492875
Nagahara H, Sonoda T, Endo K, Sugiyama Y, Taniguchi RI. High-speed imaging using CMOS image sensor with quasi pixel-wise exposure. In 2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2016. 7492875. (2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings). https://doi.org/10.1109/ICCPHOT.2016.7492875
Nagahara, Hajime ; Sonoda, Toshiki ; Endo, Kenta ; Sugiyama, Yukinobu ; Taniguchi, Rin Ichiro. / High-speed imaging using CMOS image sensor with quasi pixel-wise exposure. 2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2016. (2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings).
@inproceedings{498203c2fa704cb281e33ea72575644c,
title = "High-speed imaging using CMOS image sensor with quasi pixel-wise exposure",
abstract = "Several recent studies in compressive video sensing have realized scene capture beyond the fundamental trade-off limit between spatial resolution and temporal resolution using random space-time sampling. However, most of these studies showed results for higher frame rate video that were produced by simulation experiments or using an optically simulated random sampling camera, because there are currently no commercially available image sensors with random exposure or sampling capabilities. We fabricated a prototype complementary metal oxide semiconductor (CMOS) image sensor with quasi pixel-wise exposure timing that can realize nonuniform space-time sampling. The prototype sensor can reset exposures independently by columns and fix these amount of exposure by rows for each 8×8 pixel block. This CMOS sensor is not fully controllable via the pixels, and has line-dependent controls, but it offers flexibility when compared with regular CMOS or charge-coupled device sensors with global or rolling shutters. We propose a method to realize pseudo-random sampling for high-speed video acquisition that uses the flexibility of the CMOS sensor. We reconstruct the high-speed video sequence from the images produced by pseudo-random sampling using an over-complete dictionary. The proposed method also removes the rolling shutter effect from the reconstructed video.",
author = "Hajime Nagahara and Toshiki Sonoda and Kenta Endo and Yukinobu Sugiyama and Taniguchi, {Rin Ichiro}",
year = "2016",
month = "6",
day = "15",
doi = "10.1109/ICCPHOT.2016.7492875",
language = "English",
series = "2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings",
address = "United States",

}

TY - GEN

T1 - High-speed imaging using CMOS image sensor with quasi pixel-wise exposure

AU - Nagahara, Hajime

AU - Sonoda, Toshiki

AU - Endo, Kenta

AU - Sugiyama, Yukinobu

AU - Taniguchi, Rin Ichiro

PY - 2016/6/15

Y1 - 2016/6/15

N2 - Several recent studies in compressive video sensing have realized scene capture beyond the fundamental trade-off limit between spatial resolution and temporal resolution using random space-time sampling. However, most of these studies showed results for higher frame rate video that were produced by simulation experiments or using an optically simulated random sampling camera, because there are currently no commercially available image sensors with random exposure or sampling capabilities. We fabricated a prototype complementary metal oxide semiconductor (CMOS) image sensor with quasi pixel-wise exposure timing that can realize nonuniform space-time sampling. The prototype sensor can reset exposures independently by columns and fix these amount of exposure by rows for each 8×8 pixel block. This CMOS sensor is not fully controllable via the pixels, and has line-dependent controls, but it offers flexibility when compared with regular CMOS or charge-coupled device sensors with global or rolling shutters. We propose a method to realize pseudo-random sampling for high-speed video acquisition that uses the flexibility of the CMOS sensor. We reconstruct the high-speed video sequence from the images produced by pseudo-random sampling using an over-complete dictionary. The proposed method also removes the rolling shutter effect from the reconstructed video.

AB - Several recent studies in compressive video sensing have realized scene capture beyond the fundamental trade-off limit between spatial resolution and temporal resolution using random space-time sampling. However, most of these studies showed results for higher frame rate video that were produced by simulation experiments or using an optically simulated random sampling camera, because there are currently no commercially available image sensors with random exposure or sampling capabilities. We fabricated a prototype complementary metal oxide semiconductor (CMOS) image sensor with quasi pixel-wise exposure timing that can realize nonuniform space-time sampling. The prototype sensor can reset exposures independently by columns and fix these amount of exposure by rows for each 8×8 pixel block. This CMOS sensor is not fully controllable via the pixels, and has line-dependent controls, but it offers flexibility when compared with regular CMOS or charge-coupled device sensors with global or rolling shutters. We propose a method to realize pseudo-random sampling for high-speed video acquisition that uses the flexibility of the CMOS sensor. We reconstruct the high-speed video sequence from the images produced by pseudo-random sampling using an over-complete dictionary. The proposed method also removes the rolling shutter effect from the reconstructed video.

UR - http://www.scopus.com/inward/record.url?scp=84979705713&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84979705713&partnerID=8YFLogxK

U2 - 10.1109/ICCPHOT.2016.7492875

DO - 10.1109/ICCPHOT.2016.7492875

M3 - Conference contribution

AN - SCOPUS:84979705713

T3 - 2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings

BT - 2016 IEEE International Conference on Computational Photography, ICCP 2016 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -