3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

3D reconstruction of indoor scenes that contain non-rigidly moving human body using depth cameras is a task of extraordinary difficulty. Despite intensive efforts from the researchers in the 3D vision community, existing methods are still limited to reconstruct small scale scenes. This is because of the difficulty to track the camera motion when a target person moves in a totally different direction. Due to the narrow field of view (FoV) of consumer-grade red-green-blue-depth (RGB-D) cameras, a target person (generally put at about 2-3 meters from the camera) covers most of the FoV of the camera. Therefore, there are not enough features from the static background to track the motion of the camera. In this paper, we propose a system which reconstructs a moving human body and the background of an indoor scene using multiple depth cameras. Our system is composed of three Kinects that are approximately set in the same line and facing the same direction so that their FoV do not overlap (to avoid interference). Owing to this setup, we capture images of a person moving in a large scale indoor scene. The three Kinect cameras are calibrated with a robust method that uses three large non parallel planes. A moving person is detected by using human skeleton information, and is reconstructed separately from the static background. By separating the human body and the background, static 3D reconstruction can be adopted for the static background area while a method specialized for the human body area can be used to reconstruct the 3D model of the moving person. The experimental result shows the performance of proposed system for human body in a large-scale indoor scene.

Original languageEnglish
Title of host publicationProceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019
EditorsDongdong Weng, Liwei Chan, Youngho Lee, Xiaohui Liang, Nobuchika Sakata
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728115719
DOIs
Publication statusPublished - May 7 2019
Event12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019 - Ikoma, Nara, Japan
Duration: Mar 28 2019Mar 29 2019

Publication series

NameProceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019

Conference

Conference12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019
CountryJapan
CityIkoma, Nara
Period3/28/193/29/19

Fingerprint

Human Body
reconstruction
Cameras
human being
Skeleton
interference
Research Personnel
Human body
community
performance
Direction compound

All Science Journal Classification (ASJC) codes

  • Education
  • Sociology and Political Science
  • Communication
  • Marketing
  • Strategy and Management
  • Computer Networks and Communications
  • Information Systems and Management
  • Social Psychology

Cite this

Kobayashi, D., Thomas, D. G. F., Uchiyama, H., & Taniguchi, R-I. (2019). 3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras. In D. Weng, L. Chan, Y. Lee, X. Liang, & N. Sakata (Eds.), Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019 [8709280] (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/APMAR.2019.8709280

3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras. / Kobayashi, Daisuke; Thomas, Diego Gabriel Francis; Uchiyama, Hideaki; Taniguchi, Rin-Ichiro.

Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019. ed. / Dongdong Weng; Liwei Chan; Youngho Lee; Xiaohui Liang; Nobuchika Sakata. Institute of Electrical and Electronics Engineers Inc., 2019. 8709280 (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kobayashi, D, Thomas, DGF, Uchiyama, H & Taniguchi, R-I 2019, 3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras. in D Weng, L Chan, Y Lee, X Liang & N Sakata (eds), Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019., 8709280, Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019, Institute of Electrical and Electronics Engineers Inc., 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019, Ikoma, Nara, Japan, 3/28/19. https://doi.org/10.1109/APMAR.2019.8709280
Kobayashi D, Thomas DGF, Uchiyama H, Taniguchi R-I. 3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras. In Weng D, Chan L, Lee Y, Liang X, Sakata N, editors, Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019. Institute of Electrical and Electronics Engineers Inc. 2019. 8709280. (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019). https://doi.org/10.1109/APMAR.2019.8709280
Kobayashi, Daisuke ; Thomas, Diego Gabriel Francis ; Uchiyama, Hideaki ; Taniguchi, Rin-Ichiro. / 3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras. Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019. editor / Dongdong Weng ; Liwei Chan ; Youngho Lee ; Xiaohui Liang ; Nobuchika Sakata. Institute of Electrical and Electronics Engineers Inc., 2019. (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019).
@inproceedings{c913247f3d4e4d21bb89551a24d34a76,
title = "3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras",
abstract = "3D reconstruction of indoor scenes that contain non-rigidly moving human body using depth cameras is a task of extraordinary difficulty. Despite intensive efforts from the researchers in the 3D vision community, existing methods are still limited to reconstruct small scale scenes. This is because of the difficulty to track the camera motion when a target person moves in a totally different direction. Due to the narrow field of view (FoV) of consumer-grade red-green-blue-depth (RGB-D) cameras, a target person (generally put at about 2-3 meters from the camera) covers most of the FoV of the camera. Therefore, there are not enough features from the static background to track the motion of the camera. In this paper, we propose a system which reconstructs a moving human body and the background of an indoor scene using multiple depth cameras. Our system is composed of three Kinects that are approximately set in the same line and facing the same direction so that their FoV do not overlap (to avoid interference). Owing to this setup, we capture images of a person moving in a large scale indoor scene. The three Kinect cameras are calibrated with a robust method that uses three large non parallel planes. A moving person is detected by using human skeleton information, and is reconstructed separately from the static background. By separating the human body and the background, static 3D reconstruction can be adopted for the static background area while a method specialized for the human body area can be used to reconstruct the 3D model of the moving person. The experimental result shows the performance of proposed system for human body in a large-scale indoor scene.",
author = "Daisuke Kobayashi and Thomas, {Diego Gabriel Francis} and Hideaki Uchiyama and Rin-Ichiro Taniguchi",
year = "2019",
month = "5",
day = "7",
doi = "10.1109/APMAR.2019.8709280",
language = "English",
series = "Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
editor = "Dongdong Weng and Liwei Chan and Youngho Lee and Xiaohui Liang and Nobuchika Sakata",
booktitle = "Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019",
address = "United States",

}

TY - GEN

T1 - 3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras

AU - Kobayashi, Daisuke

AU - Thomas, Diego Gabriel Francis

AU - Uchiyama, Hideaki

AU - Taniguchi, Rin-Ichiro

PY - 2019/5/7

Y1 - 2019/5/7

N2 - 3D reconstruction of indoor scenes that contain non-rigidly moving human body using depth cameras is a task of extraordinary difficulty. Despite intensive efforts from the researchers in the 3D vision community, existing methods are still limited to reconstruct small scale scenes. This is because of the difficulty to track the camera motion when a target person moves in a totally different direction. Due to the narrow field of view (FoV) of consumer-grade red-green-blue-depth (RGB-D) cameras, a target person (generally put at about 2-3 meters from the camera) covers most of the FoV of the camera. Therefore, there are not enough features from the static background to track the motion of the camera. In this paper, we propose a system which reconstructs a moving human body and the background of an indoor scene using multiple depth cameras. Our system is composed of three Kinects that are approximately set in the same line and facing the same direction so that their FoV do not overlap (to avoid interference). Owing to this setup, we capture images of a person moving in a large scale indoor scene. The three Kinect cameras are calibrated with a robust method that uses three large non parallel planes. A moving person is detected by using human skeleton information, and is reconstructed separately from the static background. By separating the human body and the background, static 3D reconstruction can be adopted for the static background area while a method specialized for the human body area can be used to reconstruct the 3D model of the moving person. The experimental result shows the performance of proposed system for human body in a large-scale indoor scene.

AB - 3D reconstruction of indoor scenes that contain non-rigidly moving human body using depth cameras is a task of extraordinary difficulty. Despite intensive efforts from the researchers in the 3D vision community, existing methods are still limited to reconstruct small scale scenes. This is because of the difficulty to track the camera motion when a target person moves in a totally different direction. Due to the narrow field of view (FoV) of consumer-grade red-green-blue-depth (RGB-D) cameras, a target person (generally put at about 2-3 meters from the camera) covers most of the FoV of the camera. Therefore, there are not enough features from the static background to track the motion of the camera. In this paper, we propose a system which reconstructs a moving human body and the background of an indoor scene using multiple depth cameras. Our system is composed of three Kinects that are approximately set in the same line and facing the same direction so that their FoV do not overlap (to avoid interference). Owing to this setup, we capture images of a person moving in a large scale indoor scene. The three Kinect cameras are calibrated with a robust method that uses three large non parallel planes. A moving person is detected by using human skeleton information, and is reconstructed separately from the static background. By separating the human body and the background, static 3D reconstruction can be adopted for the static background area while a method specialized for the human body area can be used to reconstruct the 3D model of the moving person. The experimental result shows the performance of proposed system for human body in a large-scale indoor scene.

UR - http://www.scopus.com/inward/record.url?scp=85066297706&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066297706&partnerID=8YFLogxK

U2 - 10.1109/APMAR.2019.8709280

DO - 10.1109/APMAR.2019.8709280

M3 - Conference contribution

AN - SCOPUS:85066297706

T3 - Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019

BT - Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019

A2 - Weng, Dongdong

A2 - Chan, Liwei

A2 - Lee, Youngho

A2 - Liang, Xiaohui

A2 - Sakata, Nobuchika

PB - Institute of Electrical and Electronics Engineers Inc.

ER -