The regression of 3D Human Pose and Shape (HPS) from an image is becoming
increasingly accurate. This makes the results useful for downstream tasks like
human action recognition or 3D graphics. Yet, no regressor is perfect, and
accuracy can be affected by ambiguous image evidence or by poses and appearance
that are unseen during training. Most current HPS regressors, however, do not
report the confidence of their outputs, meaning that downstream tasks cannot
differentiate accurate estimates from inaccurate ones. To address this, we
develop POCO, a novel framework for training HPS regressors to estimate not
only a 3D human body, but also their confidence, in a single feed-forward pass.
Specifically, POCO estimates both the 3D body pose and a per-sample variance.
The key idea is to introduce a Dual Conditioning Strategy (DCS) for regressing
uncertainty that is highly correlated to pose reconstruction quality. The POCO
framework can be applied to any HPS regressor and here we evaluate it by
modifying HMR, PARE, and CLIFF. In all cases, training the network to reason
about uncertainty helps it learn to more accurately estimate 3D pose. While
this was not our goal, the improvement is modest but consistent. Our main
motivation is to provide uncertainty estimates for downstream tasks; we
demonstrate this in two ways: (1) We use the confidence estimates to bootstrap
HPS training. Given unlabelled image data, we take the confident estimates of a
POCO-trained regressor as pseudo ground truth. Retraining with this
automatically-curated data improves accuracy. (2) We exploit uncertainty in
video pose estimation by automatically identifying uncertain frames (e.g. due
to occlusion) and inpainting these from confident frames. Code and models will
be available for research at this https URL

提出了一种新的框架 POCO，用于训练 HPS 回归器，在单个前向传递中估计 3D 人体姿势和置信度，通过引入双重调节策略（DCS）来回归不确定性，从而帮助网络更准确地估计姿势。实验证明，在下游任务中使用置信度估计可以提高准确性，同时通过利用不确定性提高 HPS 训练的精确度。

POCO：带置信度的 3D 姿态和形状估计

POCO: 3D Pose and Shape Estimation with Confidence

Although the estimation of 3D human pose and shape (HPS) is rapidly
progressing, current methods still cannot reliably estimate moving humans in
global coordinates, which is critical for many applications. This is
particularly challenging when the camera is also moving, entangling human and
camera motion. To address these issues, we adopt a novel 5D representation
(space, time, and identity) that enables end-to-end reasoning about people in
scenes. Our method, called TRACE, introduces several novel architectural
components. Most importantly, it uses two new "maps" to reason about the 3D
trajectory of people over time in camera, and world, coordinates. An additional
memory unit enables persistent tracking of people even during long occlusions.
TRACE is the first one-stage method to jointly recover and track 3D humans in
global coordinates from dynamic cameras. By training it end-to-end, and using
full image information, TRACE achieves state-of-the-art performance on tracking
and HPS benchmarks. The code and dataset are released for research purposes.

TRACE 是一种使用 5D 表示法并引入了两个新 “地图” 以推理关于动态摄像机下人物全局坐标中 3D 轨迹的方法，使用端到端模型训练，实现了在跟踪和 HPS 基准上的最先进性能。

5D 时态回归：动态相机下的 3D 环境阿凡达

TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D  Environments

Due to the lack of camera parameter information for in-the-wild images,
existing 3D human pose and shape (HPS) estimation methods make several
simplifying assumptions: weak-perspective projection, large constant focal
length, and zero camera rotation. These assumptions often do not hold and we
show, quantitatively and qualitatively, that they cause errors in the
reconstructed 3D shape and pose. To address this, we introduce SPEC, the first
in-the-wild 3D HPS method that estimates the perspective camera from a single
image and employs this to reconstruct 3D human bodies more accurately. First,
we train a neural network to estimate the field of view, camera pitch, and roll
given an input image. We employ novel losses that improve the calibration
accuracy over previous work. We then train a novel network that concatenates
the camera calibration to the image features and uses these together to regress
3D body shape and pose. SPEC is more accurate than the prior art on the
standard benchmark (3DPW) as well as two new datasets with more challenging
camera views and varying focal lengths. Specifically, we create a new
photorealistic synthetic dataset (SPEC-SYN) with ground truth 3D bodies and a
novel in-the-wild dataset (SPEC-MTP) with calibration and high-quality
reference bodies. Both qualitative and quantitative analysis confirm that
knowing camera parameters during inference regresses better human bodies. Code
and datasets are available for research purposes at this https URL

SPEC 是第一个在野外图像中从单幅图像估计透视相机并更准确地重构 3D 人体的 3D 人体姿态和形状方法，SPEC 通过训练神经网络来估计视场角、相机俯仰角和滚转角，并将相机标定与图像特征连接起来，一起用于回归 3D 人体的形状和姿势，其结果在标准基准和两个新的数据集中表现比之前的艺术更加准确。