Human Shape Reconstruction with Loose Clothes from Partially Observed Data by Pose Specific Deformation

Akihiko Sayo, Hayato Onizuka, Diego Thomas, Yuta Nakashima, Hiroshi Kawasaki, Katsushi Ikeuchi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Reconstructing the entire body of moving human in a computer is important for various applications, such as tele-presence, virtual try-on, etc. For the purpose, realistic representation of loose clothes or non-rigid body deformation is a challenging and important task. Recent approaches for full-body reconstruction use a statistical shape model, which is built upon accurate full-body scans of people in skin-tight clothes. Such a model can be fitted to a point cloud of a person wearing loose clothes, however, it cannot represent the detailed shape of loose clothes, such as wrinkles and/or folds. In this paper, we propose a method that reconstructs 3D model of full-body human with loose clothes by reproducing the deformations as displacements from the skin-tight body mesh. We take advantage of a statistical shape model as base shape of full-body human mesh, and then, obtain displacements from the base mesh by non-rigid registration. To efficiently represent such displacements, we use lower dimensional embeddings of the deformations. This enables us to regress the coefficients corresponding to the small number of bases. We also propose a method to reconstruct shape only from a single 3D scanner, which is realized by shape fitting to only visible meshes as well as intra-frame shape interpolation. Our experiments with both unknown scene and partial body scans confirm the reconstruction ability of our proposed method.

Original languageEnglish
Title of host publicationImage and Video Technology - 9th Pacific-Rim Symposium, PSIVT 2019, Proceedings
EditorsChilwoo Lee, Zhixun Su, Akihiro Sugimoto
PublisherSpringer
Pages225-239
Number of pages15
ISBN (Print)9783030348786
DOIs
Publication statusPublished - Jan 1 2019
Event9th Pacific-Rim Symposium on Image and Video Technology, PSIVT 2019 - Sydney, Australia
Duration: Nov 18 2019Nov 22 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11854 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference9th Pacific-Rim Symposium on Image and Video Technology, PSIVT 2019
CountryAustralia
CitySydney
Period11/18/1911/22/19

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Human Shape Reconstruction with Loose Clothes from Partially Observed Data by Pose Specific Deformation'. Together they form a unique fingerprint.

  • Cite this

    Sayo, A., Onizuka, H., Thomas, D., Nakashima, Y., Kawasaki, H., & Ikeuchi, K. (2019). Human Shape Reconstruction with Loose Clothes from Partially Observed Data by Pose Specific Deformation. In C. Lee, Z. Su, & A. Sugimoto (Eds.), Image and Video Technology - 9th Pacific-Rim Symposium, PSIVT 2019, Proceedings (pp. 225-239). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11854 LNCS). Springer. https://doi.org/10.1007/978-3-030-34879-3_18