TY - GEN
T1 - RGB-D Images Based 3D Plant Growth Prediction by Sequential Images-to-Images Translation with Plant Priors
AU - Hamamoto, Tomohiro
AU - Uchiyama, Hideaki
AU - Shimada, Atsushi
AU - Taniguchi, Rin ichiro
N1 - Funding Information:
Supported by JSPS KAKENHI Grant Number JP17H01768 and JP18H04117.
Publisher Copyright:
© 2022, Springer Nature Switzerland AG.
PY - 2022
Y1 - 2022
N2 - This paper presents a neural network based method for 3D plant growth prediction based on sequential images-to-images translation. Especially, we extend an existing image-to-image translation technique based on U-Net to images-to-images translation by incorporating convLSTM into skip connections in U-Net. With this architecture, we can achieve sequential image prediction tasks such that future images are predicted from several past ones. Since depth images are incorporated as additional channel into our network, the prediction can be represented in 3D space. As an application of our method, we develop a 3D plant growth prediction system. In the evaluation, the performance of our network was investigated in terms of the importance of each module in the network. We verified how the prediction accuracy was affected by the internal structure of the network. In addition, the extension of our network with plant priors was further investigated to evaluate the impact for plant growth prediction tasks.
AB - This paper presents a neural network based method for 3D plant growth prediction based on sequential images-to-images translation. Especially, we extend an existing image-to-image translation technique based on U-Net to images-to-images translation by incorporating convLSTM into skip connections in U-Net. With this architecture, we can achieve sequential image prediction tasks such that future images are predicted from several past ones. Since depth images are incorporated as additional channel into our network, the prediction can be represented in 3D space. As an application of our method, we develop a 3D plant growth prediction system. In the evaluation, the performance of our network was investigated in terms of the importance of each module in the network. We verified how the prediction accuracy was affected by the internal structure of the network. In addition, the extension of our network with plant priors was further investigated to evaluate the impact for plant growth prediction tasks.
UR - http://www.scopus.com/inward/record.url?scp=85124654006&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85124654006&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-94893-1_15
DO - 10.1007/978-3-030-94893-1_15
M3 - Conference contribution
AN - SCOPUS:85124654006
SN - 9783030948924
T3 - Communications in Computer and Information Science
SP - 334
EP - 352
BT - Computer Vision, Imaging and Computer Graphics Theory and Applications - 15th International Joint Conference, VISIGRAPP 2020, Revised Selected Papers
A2 - Bouatouch, Kadi
A2 - de Sousa, A. Augusto
A2 - Chessa, Manuela
A2 - Paljic, Alexis
A2 - Kerren, Andreas
A2 - Hurter, Christophe
A2 - Farinella, Giovanni Maria
A2 - Radeva, Petia
A2 - Braz, Jose
PB - Springer Science and Business Media Deutschland GmbH
T2 - 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISIGRAPP 2020
Y2 - 27 February 2020 through 29 February 2020
ER -