Unsupervised Multi-view Multi-person 3D Pose Estimation Using Reprojection Error

Diógenes Wallis de França Silva, João Paulo Silva do Monte Lima, David Macêdo, Cleber Zanchettin, Diego Gabriel Francis Thomas, Hideaki Uchiyama, Veronica Teichrieb

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This work addresses multi-view multi-person 3D pose estimation in synchronized and calibrated camera views. Recent approaches estimate neural network weights in a supervised way; they rely on ground truth annotated datasets to compute the loss function and optimize the weights in the network. However, manually labeling ground truth datasets is labor-intensive, expensive, and prone to errors. Consequently, it is preferable not to rely heavily on labeled datasets. This work proposes an unsupervised approach to estimating 3D human poses requiring only an off-the-shelf 2D pose estimation method and the intrinsic and extrinsic camera parameters. Our approach uses reprojection error as a loss function instead of comparing the predicted 3D pose with the ground truth. First, we estimate the 3D pose of each person using the plane sweep stereo approach, in which the depth of each 2D joint related to each person is estimated in a selected target view. The estimated 3D pose is then projected onto each of the other views using camera parameters. Finally, the 2D reprojection error in the image plane is computed by comparing it with the estimated 2D pose corresponding to the same person. The 2D poses that correspond to the same person are identified using virtual depth planes, where each 3D pose is projected onto the reference view and compared to find the nearest 2D pose. Our proposed method learns to estimate 3D pose in an end-to-end unsupervised manner and does not require any manual parameter tuning, yet we achieved results close to state-of-the-art supervised methods on a public dataset. Our method achieves only 5.8% points below the fully supervised state-of-the-art method and only 5.1% points below the best geometric approach in the Campus dataset.

Original languageEnglish
Title of host publicationArtificial Neural Networks and Machine Learning – ICANN 2022 - 31st International Conference on Artificial Neural Networks, 2022, Proceedings
EditorsElias Pimenidis, Plamen Angelov, Chrisina Jayne, Antonios Papaleonidas, Mehmet Aydin
PublisherSpringer Science and Business Media Deutschland GmbH
Pages482-494
Number of pages13
ISBN (Print)9783031159336
DOIs
Publication statusPublished - 2022
Event31st International Conference on Artificial Neural Networks, ICANN 2022 - Bristol, United Kingdom
Duration: Sep 6 2022Sep 9 2022

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13531 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference31st International Conference on Artificial Neural Networks, ICANN 2022
Country/TerritoryUnited Kingdom
CityBristol
Period9/6/229/9/22

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Unsupervised Multi-view Multi-person 3D Pose Estimation Using Reprojection Error'. Together they form a unique fingerprint.

Cite this