Realtime novel view synthesis with eigen-texture regression

Yuta Nakashima, Fumio Okura, Norihiko Kawai, Hiroshi Kawasaki, Ambrosio Blanco, Katsushi Ikeuchi

研究成果: 書籍/レポート タイプへの寄稿会議への寄与

4 被引用数 (Scopus)


Realtime novel view synthesis, which generates a novel view of a real object or scene in realtime, enjoys a wide range of applications including augmented reality, telepresence, and immersive telecommunication. Image-based rendering (IBR) with rough geometry can be done using only an off-the-shelf camera and thus can be used by many users. However, IBR from images in the wild (e.g., lighting condition changes or the scene contains objects with specular surfaces) has been a tough problem due to color discontinuity; IBR with rough geometry picks up appropriate images for a given viewpoint, but the image used for a rendering unit (a face or pixel) switches when the viewpoint moves, which may cause noticeable changes in color. We use the eigen-texture technique, which represents images for a certain face using a point in the eigenspace. We propose to regress a new point in this space, which moves smoothly, given a viewpoint so that we can generate an image whose color smoothly changes according to the point. Our regressor is based on a neural network with a single hidden layer and hyperbolic tangent nonlinearity. We demonstrate the advantages of our IBR approach using our own datasets as well as publicly available datasets for comparison.

ホスト出版物のタイトルBritish Machine Vision Conference 2017, BMVC 2017
出版社BMVA Press
ISBN(電子版)190172560X, 9781901725605
出版ステータス出版済み - 2017
イベント28th British Machine Vision Conference, BMVC 2017 - London, 英国
継続期間: 9月 4 20179月 7 2017


名前British Machine Vision Conference 2017, BMVC 2017


会議28th British Machine Vision Conference, BMVC 2017

!!!All Science Journal Classification (ASJC) codes

  • コンピュータ ビジョンおよびパターン認識


「Realtime novel view synthesis with eigen-texture regression」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。