TY - JOUR
T1 - Virtual IR Sensing for Planetary Rovers
T2 - Improved Terrain Classification and Thermal Inertia Estimation
AU - Iwashita, Yumi
AU - Nakashima, Kazuto
AU - Gatto, Joseph
AU - Higa, Shoya
AU - Stoica, Adrian
AU - Khoo, Norris
AU - Kurazume, Ryo
N1 - Funding Information:
Manuscript received February 24, 2020; accepted July 19, 2020. Date of publication August 4, 2020; date of current version August 11, 2020. This letter was recommended for publication by Associate Editor N. Kottege and Editor J. Roberts upon evaluation of the reviewers’ comments. This work was supported by the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. (Corresponding author: Yumi Iwashita.) Yumi Iwashita, Shoya Higa, and Adrian Stoica are with the Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109 USA (e-mail: yumi.iwashita@jpl.nasa.gov; shoya.higa@jpl.nasa.gov; adrian.stoica@jpl.nasa.gov).
Publisher Copyright:
© 2016 IEEE.
PY - 2020/10
Y1 - 2020/10
N2 - Terrain classification is critically important for Mars rovers, which rely on it for planning and autonomous navigation. On-board terrain classification using visual information has limitations, and is sensitive to illumination conditions. Classification can be improved if one fuses visual imagery with additional infrared (IR) imagery of the scene, yet unfortunately there are no IR image sensors on the current Mars rovers. A virtual IR sensor, estimating IR from RGB imagery using deep learning, was proposed in the context of a MU-Net architecture. However, virtual IR estimation was limited by the fact that slope angle variations induce temperature differences within the same terrain. This paper removes this limitation, giving good IR estimates and as a consequence improving terrain classification by including the additional angle from the surface normal to the Sun and the measurement of solar radiation. The estimates are also useful when estimating thermal inertia, which can enhance slip prediction and small rock density estimation. Our approach is demonstrated in two applications. We collected a new data set to verify the effectiveness of the proposed approach and show its benefit by applying to the two applications.
AB - Terrain classification is critically important for Mars rovers, which rely on it for planning and autonomous navigation. On-board terrain classification using visual information has limitations, and is sensitive to illumination conditions. Classification can be improved if one fuses visual imagery with additional infrared (IR) imagery of the scene, yet unfortunately there are no IR image sensors on the current Mars rovers. A virtual IR sensor, estimating IR from RGB imagery using deep learning, was proposed in the context of a MU-Net architecture. However, virtual IR estimation was limited by the fact that slope angle variations induce temperature differences within the same terrain. This paper removes this limitation, giving good IR estimates and as a consequence improving terrain classification by including the additional angle from the surface normal to the Sun and the measurement of solar radiation. The estimates are also useful when estimating thermal inertia, which can enhance slip prediction and small rock density estimation. Our approach is demonstrated in two applications. We collected a new data set to verify the effectiveness of the proposed approach and show its benefit by applying to the two applications.
UR - http://www.scopus.com/inward/record.url?scp=85089946506&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85089946506&partnerID=8YFLogxK
U2 - 10.1109/LRA.2020.3013909
DO - 10.1109/LRA.2020.3013909
M3 - Article
AN - SCOPUS:85089946506
VL - 5
SP - 6302
EP - 6309
JO - IEEE Robotics and Automation Letters
JF - IEEE Robotics and Automation Letters
SN - 2377-3766
IS - 4
M1 - 9158384
ER -