Reflectance and Shape Estimation with a Light Field Camera Under Natural Illumination

Thanh Trung Ngo, Hajime Nagahara, Ko Nishino, Rin ichiro Taniguchi, Yasushi Yagi

Research output: Contribution to journalArticle

Abstract

Reflectance and shape are two important components in visually perceiving the real world. Inferring the reflectance and shape of an object through cameras is a fundamental research topic in the field of computer vision. While three-dimensional shape recovery is pervasive with varieties of approaches and practical applications, reflectance recovery has only emerged recently. Reflectance recovery is a challenging task that is usually conducted in controlled environments, such as a laboratory environment with a special apparatus. However, it is desirable that the reflectance be recovered in the field with a handy camera so that reflectance can be jointly recovered with the shape. To that end, we present a solution that simultaneously recovers the reflectance and shape (i.e., dense depth and normal maps) of an object under natural illumination with commercially available handy cameras. We employ a light field camera to capture one light field image of the object, and a 360-degree camera to capture the illumination. The proposed method provides positive results in both simulation and real-world experiments.

Original languageEnglish
Pages (from-to)1707-1722
Number of pages16
JournalInternational Journal of Computer Vision
Volume127
Issue number11-12
DOIs
Publication statusPublished - Dec 1 2019

Fingerprint

Lighting
Cameras
Recovery
Computer vision
Experiments

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

Reflectance and Shape Estimation with a Light Field Camera Under Natural Illumination. / Ngo, Thanh Trung; Nagahara, Hajime; Nishino, Ko; Taniguchi, Rin ichiro; Yagi, Yasushi.

In: International Journal of Computer Vision, Vol. 127, No. 11-12, 01.12.2019, p. 1707-1722.

Research output: Contribution to journalArticle

Ngo, Thanh Trung ; Nagahara, Hajime ; Nishino, Ko ; Taniguchi, Rin ichiro ; Yagi, Yasushi. / Reflectance and Shape Estimation with a Light Field Camera Under Natural Illumination. In: International Journal of Computer Vision. 2019 ; Vol. 127, No. 11-12. pp. 1707-1722.
@article{89458e803923442e8e66128505c9a66a,
title = "Reflectance and Shape Estimation with a Light Field Camera Under Natural Illumination",
abstract = "Reflectance and shape are two important components in visually perceiving the real world. Inferring the reflectance and shape of an object through cameras is a fundamental research topic in the field of computer vision. While three-dimensional shape recovery is pervasive with varieties of approaches and practical applications, reflectance recovery has only emerged recently. Reflectance recovery is a challenging task that is usually conducted in controlled environments, such as a laboratory environment with a special apparatus. However, it is desirable that the reflectance be recovered in the field with a handy camera so that reflectance can be jointly recovered with the shape. To that end, we present a solution that simultaneously recovers the reflectance and shape (i.e., dense depth and normal maps) of an object under natural illumination with commercially available handy cameras. We employ a light field camera to capture one light field image of the object, and a 360-degree camera to capture the illumination. The proposed method provides positive results in both simulation and real-world experiments.",
author = "Ngo, {Thanh Trung} and Hajime Nagahara and Ko Nishino and Taniguchi, {Rin ichiro} and Yasushi Yagi",
year = "2019",
month = "12",
day = "1",
doi = "10.1007/s11263-019-01149-5",
language = "English",
volume = "127",
pages = "1707--1722",
journal = "International Journal of Computer Vision",
issn = "0920-5691",
publisher = "Springer Netherlands",
number = "11-12",

}

TY - JOUR

T1 - Reflectance and Shape Estimation with a Light Field Camera Under Natural Illumination

AU - Ngo, Thanh Trung

AU - Nagahara, Hajime

AU - Nishino, Ko

AU - Taniguchi, Rin ichiro

AU - Yagi, Yasushi

PY - 2019/12/1

Y1 - 2019/12/1

N2 - Reflectance and shape are two important components in visually perceiving the real world. Inferring the reflectance and shape of an object through cameras is a fundamental research topic in the field of computer vision. While three-dimensional shape recovery is pervasive with varieties of approaches and practical applications, reflectance recovery has only emerged recently. Reflectance recovery is a challenging task that is usually conducted in controlled environments, such as a laboratory environment with a special apparatus. However, it is desirable that the reflectance be recovered in the field with a handy camera so that reflectance can be jointly recovered with the shape. To that end, we present a solution that simultaneously recovers the reflectance and shape (i.e., dense depth and normal maps) of an object under natural illumination with commercially available handy cameras. We employ a light field camera to capture one light field image of the object, and a 360-degree camera to capture the illumination. The proposed method provides positive results in both simulation and real-world experiments.

AB - Reflectance and shape are two important components in visually perceiving the real world. Inferring the reflectance and shape of an object through cameras is a fundamental research topic in the field of computer vision. While three-dimensional shape recovery is pervasive with varieties of approaches and practical applications, reflectance recovery has only emerged recently. Reflectance recovery is a challenging task that is usually conducted in controlled environments, such as a laboratory environment with a special apparatus. However, it is desirable that the reflectance be recovered in the field with a handy camera so that reflectance can be jointly recovered with the shape. To that end, we present a solution that simultaneously recovers the reflectance and shape (i.e., dense depth and normal maps) of an object under natural illumination with commercially available handy cameras. We employ a light field camera to capture one light field image of the object, and a 360-degree camera to capture the illumination. The proposed method provides positive results in both simulation and real-world experiments.

UR - http://www.scopus.com/inward/record.url?scp=85061361639&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061361639&partnerID=8YFLogxK

U2 - 10.1007/s11263-019-01149-5

DO - 10.1007/s11263-019-01149-5

M3 - Article

AN - SCOPUS:85061361639

VL - 127

SP - 1707

EP - 1722

JO - International Journal of Computer Vision

JF - International Journal of Computer Vision

SN - 0920-5691

IS - 11-12

ER -