Monocular Human Pose Estimation (HPE) aims at determining the 3D positions of
human joints from a single 2D image captured by a camera. However, a single 2D
point in the image may correspond to multiple points in 3D space. Typically,
the uniqueness of the 2D-3D relationship is approximated using an orthographic
or weak-perspective camera model. In this study, instead of relying on
approximations, we advocate for utilizing the full perspective camera model.
This involves estimating camera parameters and establishing a precise,
unambiguous 2D-3D relationship. To do so, we introduce the EPOCH framework,
comprising two main components: the pose lifter network (LiftNet) and the pose
regressor network (RegNet). LiftNet utilizes the full perspective camera model
to precisely estimate the 3D pose in an unsupervised manner. It takes a 2D pose
and camera parameters as inputs and produces the corresponding 3D pose
estimation. These inputs are obtained from RegNet, which starts from a single
image and provides estimates for the 2D pose and camera parameters. RegNet
utilizes only 2D pose data as weak supervision. Internally, RegNet predicts a
3D pose, which is then projected to 2D using the estimated camera parameters.
This process enables RegNet to establish the unambiguous 2D-3D relationship.
Our experiments show that modeling the lifting as an unsupervised task with a
camera in-the-loop results in better generalization to unseen data. We obtain
state-of-the-art results for the 3D HPE on the Human3.6M and MPI-INF-3DHP
datasets. Our code is available at: [Github link upon acceptance, see
supplementary materials].

单目人体姿态估计（HPE）是从相机捕捉的单个 2D 图像中确定人体关节的 3D 位置。本研究提出了一种 EPOCH 框架，利用完整的透视相机模型进行姿态估计，并通过无监督学习得到比现有方法更好的泛化性能和最先进的结果。

EPOCH: 估计相机和人类的三维姿态

EPOCH: Jointly Estimating the 3D Pose of Cameras and Humans

Humans live within a 3D space and constantly interact with it to perform
tasks. Such interactions involve physical contact between surfaces that is
semantically meaningful. Our goal is to learn how humans interact with scenes
and leverage this to enable virtual characters to do the same. To that end, we
introduce a novel Human-Scene Interaction (HSI) model that encodes proximal
relationships, called POSA for "Pose with prOximitieS and contActs". The
representation of interaction is body-centric, which enables it to generalize
to new scenes. Specifically, POSA augments the SMPL-X parametric human body
model such that, for every mesh vertex, it encodes (a) the contact probability
with the scene surface and (b) the corresponding semantic scene label. We learn
POSA with a VAE conditioned on the SMPL-X vertices, and train on the PROX
dataset, which contains SMPL-X meshes of people interacting with 3D scenes, and
the corresponding scene semantics from the PROX-E dataset. We demonstrate the
value of POSA with two applications. First, we automatically place 3D scans of
people in scenes. We use a SMPL-X model fit to the scan as a proxy and then
find its most likely placement in 3D. POSA provides an effective representation
to search for "affordances" in the scene that match the likely contact
relationships for that pose. We perform a perceptual study that shows
significant improvement over the state of the art on this task. Second, we show
that POSA's learned representation of body-scene interaction supports monocular
human pose estimation that is consistent with a 3D scene, improving on the
state of the art. Our model and code are available for research purposes at
this https URL

本文提出了 POSA 模型，用来学习人体与场景的交互，包括接触概率和语义场景标签，并展示了其在 3D 人物自动放置和一致的单目人体姿态估计方面的改进。