3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras

研究成果: 著書/レポートタイプへの貢献会議での発言

抄録

3D reconstruction of indoor scenes that contain non-rigidly moving human body using depth cameras is a task of extraordinary difficulty. Despite intensive efforts from the researchers in the 3D vision community, existing methods are still limited to reconstruct small scale scenes. This is because of the difficulty to track the camera motion when a target person moves in a totally different direction. Due to the narrow field of view (FoV) of consumer-grade red-green-blue-depth (RGB-D) cameras, a target person (generally put at about 2-3 meters from the camera) covers most of the FoV of the camera. Therefore, there are not enough features from the static background to track the motion of the camera. In this paper, we propose a system which reconstructs a moving human body and the background of an indoor scene using multiple depth cameras. Our system is composed of three Kinects that are approximately set in the same line and facing the same direction so that their FoV do not overlap (to avoid interference). Owing to this setup, we capture images of a person moving in a large scale indoor scene. The three Kinect cameras are calibrated with a robust method that uses three large non parallel planes. A moving person is detected by using human skeleton information, and is reconstructed separately from the static background. By separating the human body and the background, static 3D reconstruction can be adopted for the static background area while a method specialized for the human body area can be used to reconstruct the 3D model of the moving person. The experimental result shows the performance of proposed system for human body in a large-scale indoor scene.

元の言語英語
ホスト出版物のタイトルProceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019
編集者Dongdong Weng, Liwei Chan, Youngho Lee, Xiaohui Liang, Nobuchika Sakata
出版者Institute of Electrical and Electronics Engineers Inc.
ISBN(電子版)9781728115719
DOI
出版物ステータス出版済み - 5 7 2019
イベント12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019 - Ikoma, Nara, 日本
継続期間: 3 28 20193 29 2019

出版物シリーズ

名前Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019

会議

会議12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019
日本
Ikoma, Nara
期間3/28/193/29/19

Fingerprint

Human Body
reconstruction
Cameras
human being
Skeleton
interference
Research Personnel
Human body
community
performance
Direction compound

All Science Journal Classification (ASJC) codes

  • Education
  • Sociology and Political Science
  • Communication
  • Marketing
  • Strategy and Management
  • Computer Networks and Communications
  • Information Systems and Management
  • Social Psychology

これを引用

Kobayashi, D., Thomas, D. G. F., Uchiyama, H., & Taniguchi, R-I. (2019). 3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras. : D. Weng, L. Chan, Y. Lee, X. Liang, & N. Sakata (版), Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019 [8709280] (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/APMAR.2019.8709280

3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras. / Kobayashi, Daisuke; Thomas, Diego Gabriel Francis; Uchiyama, Hideaki; Taniguchi, Rin-Ichiro.

Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019. 版 / Dongdong Weng; Liwei Chan; Youngho Lee; Xiaohui Liang; Nobuchika Sakata. Institute of Electrical and Electronics Engineers Inc., 2019. 8709280 (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019).

研究成果: 著書/レポートタイプへの貢献会議での発言

Kobayashi, D, Thomas, DGF, Uchiyama, H & Taniguchi, R-I 2019, 3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras. : D Weng, L Chan, Y Lee, X Liang & N Sakata (版), Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019., 8709280, Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019, Institute of Electrical and Electronics Engineers Inc., 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019, Ikoma, Nara, 日本, 3/28/19. https://doi.org/10.1109/APMAR.2019.8709280
Kobayashi D, Thomas DGF, Uchiyama H, Taniguchi R-I. 3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras. : Weng D, Chan L, Lee Y, Liang X, Sakata N, 編集者, Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019. Institute of Electrical and Electronics Engineers Inc. 2019. 8709280. (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019). https://doi.org/10.1109/APMAR.2019.8709280
Kobayashi, Daisuke ; Thomas, Diego Gabriel Francis ; Uchiyama, Hideaki ; Taniguchi, Rin-Ichiro. / 3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras. Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019. 編集者 / Dongdong Weng ; Liwei Chan ; Youngho Lee ; Xiaohui Liang ; Nobuchika Sakata. Institute of Electrical and Electronics Engineers Inc., 2019. (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019).
@inproceedings{c913247f3d4e4d21bb89551a24d34a76,
title = "3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras",
abstract = "3D reconstruction of indoor scenes that contain non-rigidly moving human body using depth cameras is a task of extraordinary difficulty. Despite intensive efforts from the researchers in the 3D vision community, existing methods are still limited to reconstruct small scale scenes. This is because of the difficulty to track the camera motion when a target person moves in a totally different direction. Due to the narrow field of view (FoV) of consumer-grade red-green-blue-depth (RGB-D) cameras, a target person (generally put at about 2-3 meters from the camera) covers most of the FoV of the camera. Therefore, there are not enough features from the static background to track the motion of the camera. In this paper, we propose a system which reconstructs a moving human body and the background of an indoor scene using multiple depth cameras. Our system is composed of three Kinects that are approximately set in the same line and facing the same direction so that their FoV do not overlap (to avoid interference). Owing to this setup, we capture images of a person moving in a large scale indoor scene. The three Kinect cameras are calibrated with a robust method that uses three large non parallel planes. A moving person is detected by using human skeleton information, and is reconstructed separately from the static background. By separating the human body and the background, static 3D reconstruction can be adopted for the static background area while a method specialized for the human body area can be used to reconstruct the 3D model of the moving person. The experimental result shows the performance of proposed system for human body in a large-scale indoor scene.",
author = "Daisuke Kobayashi and Thomas, {Diego Gabriel Francis} and Hideaki Uchiyama and Rin-Ichiro Taniguchi",
year = "2019",
month = "5",
day = "7",
doi = "10.1109/APMAR.2019.8709280",
language = "English",
series = "Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
editor = "Dongdong Weng and Liwei Chan and Youngho Lee and Xiaohui Liang and Nobuchika Sakata",
booktitle = "Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019",
address = "United States",

}

TY - GEN

T1 - 3D Body and Background Reconstruction in a Large-scale Indoor Scene using Multiple Depth Cameras

AU - Kobayashi, Daisuke

AU - Thomas, Diego Gabriel Francis

AU - Uchiyama, Hideaki

AU - Taniguchi, Rin-Ichiro

PY - 2019/5/7

Y1 - 2019/5/7

N2 - 3D reconstruction of indoor scenes that contain non-rigidly moving human body using depth cameras is a task of extraordinary difficulty. Despite intensive efforts from the researchers in the 3D vision community, existing methods are still limited to reconstruct small scale scenes. This is because of the difficulty to track the camera motion when a target person moves in a totally different direction. Due to the narrow field of view (FoV) of consumer-grade red-green-blue-depth (RGB-D) cameras, a target person (generally put at about 2-3 meters from the camera) covers most of the FoV of the camera. Therefore, there are not enough features from the static background to track the motion of the camera. In this paper, we propose a system which reconstructs a moving human body and the background of an indoor scene using multiple depth cameras. Our system is composed of three Kinects that are approximately set in the same line and facing the same direction so that their FoV do not overlap (to avoid interference). Owing to this setup, we capture images of a person moving in a large scale indoor scene. The three Kinect cameras are calibrated with a robust method that uses three large non parallel planes. A moving person is detected by using human skeleton information, and is reconstructed separately from the static background. By separating the human body and the background, static 3D reconstruction can be adopted for the static background area while a method specialized for the human body area can be used to reconstruct the 3D model of the moving person. The experimental result shows the performance of proposed system for human body in a large-scale indoor scene.

AB - 3D reconstruction of indoor scenes that contain non-rigidly moving human body using depth cameras is a task of extraordinary difficulty. Despite intensive efforts from the researchers in the 3D vision community, existing methods are still limited to reconstruct small scale scenes. This is because of the difficulty to track the camera motion when a target person moves in a totally different direction. Due to the narrow field of view (FoV) of consumer-grade red-green-blue-depth (RGB-D) cameras, a target person (generally put at about 2-3 meters from the camera) covers most of the FoV of the camera. Therefore, there are not enough features from the static background to track the motion of the camera. In this paper, we propose a system which reconstructs a moving human body and the background of an indoor scene using multiple depth cameras. Our system is composed of three Kinects that are approximately set in the same line and facing the same direction so that their FoV do not overlap (to avoid interference). Owing to this setup, we capture images of a person moving in a large scale indoor scene. The three Kinect cameras are calibrated with a robust method that uses three large non parallel planes. A moving person is detected by using human skeleton information, and is reconstructed separately from the static background. By separating the human body and the background, static 3D reconstruction can be adopted for the static background area while a method specialized for the human body area can be used to reconstruct the 3D model of the moving person. The experimental result shows the performance of proposed system for human body in a large-scale indoor scene.

UR - http://www.scopus.com/inward/record.url?scp=85066297706&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066297706&partnerID=8YFLogxK

U2 - 10.1109/APMAR.2019.8709280

DO - 10.1109/APMAR.2019.8709280

M3 - Conference contribution

AN - SCOPUS:85066297706

T3 - Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019

BT - Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019

A2 - Weng, Dongdong

A2 - Chan, Liwei

A2 - Lee, Youngho

A2 - Liang, Xiaohui

A2 - Sakata, Nobuchika

PB - Institute of Electrical and Electronics Engineers Inc.

ER -