We introduce UPose3D, a novel approach for multi-view 3D human pose
estimation, addressing challenges in accuracy and scalability. Our method
advances existing pose estimation frameworks by improving robustness and
flexibility without requiring direct 3D annotations. At the core of our method,
a pose compiler module refines predictions from a 2D keypoints estimator that
operates on a single image by leveraging temporal and cross-view information.
Our novel cross-view fusion strategy is scalable to any number of cameras,
while our synthetic data generation strategy ensures generalization across
diverse actors, scenes, and viewpoints. Finally, UPose3D leverages the
prediction uncertainty of both the 2D keypoint estimator and the pose compiler
module. This provides robustness to outliers and noisy data, resulting in
state-of-the-art performance in out-of-distribution settings. In addition, for
in-distribution settings, UPose3D yields a performance rivaling methods that
rely on 3D annotated data, while being the state-of-the-art among methods
relying only on 2D supervision.

UPose3D 是一种新颖的多视图 3D 人体姿势估计方法，通过改进鲁棒性和可扩展性来解决准确性的挑战，同时不需要直接的 3D 注释。

UPose3D：带有跨视角和时间线索的不确定性感知 3D 人体姿态估计

UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and  Temporal Cues

Multi-view 3D human pose estimation is naturally superior to single view one,
benefiting from more comprehensive information provided by images of multiple
views. The information includes camera poses, 2D/3D human poses, and 3D
geometry. However, the accurate annotation of these information is hard to
obtain, making it challenging to predict accurate 3D human pose from multi-view
images. To deal with this issue, we propose a fully self-supervised framework,
named cascaded multi-view aggregating network (CMANet), to construct a
canonical parameter space to holistically integrate and exploit multi-view
information. In our framework, the multi-view information is grouped into two
categories: 1) intra-view information , 2) inter-view information. Accordingly,
CMANet consists of two components: intra-view module (IRV) and inter-view
module (IEV). IRV is used for extracting initial camera pose and 3D human pose
of each view; IEV is to fuse complementary pose information and cross-view 3D
geometry for a final 3D human pose. To facilitate the aggregation of the intra-
and inter-view, we define a canonical parameter space, depicted by per-view
camera pose and human pose and shape parameters ($\theta$ and $\beta$) of SMPL
model, and propose a two-stage learning procedure. At first stage, IRV learns
to estimate camera pose and view-dependent 3D human pose supervised by
confident output of an off-the-shelf 2D keypoint detector. At second stage, IRV
is frozen and IEV further refines the camera pose and optimizes the 3D human
pose by implicitly encoding the cross-view complement and 3D geometry
constraint, achieved by jointly fitting predicted multi-view 2D keypoints. The
proposed framework, modules, and learning strategy are demonstrated to be
effective by comprehensive experiments and CMANet is superior to
state-of-the-art methods in extensive quantitative and qualitative analysis.

提出了一种名为 CMANet 的全自我监督框架，利用多视角信息来预测准确的多视角 3D 人体姿势，通过聚合视角内和视角间信息，构建一个规范化参数空间

自学习的规范空间用于多视角三维人体姿态估计

Self-learning Canonical Space for Multi-view 3D Human Pose Estimation

In this work, we aim to improve the 3D reasoning ability of Transformers in
multi-view 3D human pose estimation. Recent works have focused on end-to-end
learning-based transformer designs, which struggle to resolve geometric
information accurately, particularly during occlusion. Instead, we propose a
novel hybrid model, MVGFormer, which has a series of geometric and appearance
modules organized in an iterative manner. The geometry modules are
learning-free and handle all viewpoint-dependent 3D tasks geometrically which
notably improves the model's generalization ability. The appearance modules are
learnable and are dedicated to estimating 2D poses from image signals
end-to-end which enables them to achieve accurate estimates even when occlusion
occurs, leading to a model that is both accurate and generalizable to new
cameras and geometries. We evaluate our approach for both in-domain and
out-of-domain settings, where our model consistently outperforms
state-of-the-art methods, and especially does so by a significant margin in the
out-of-domain setting. We will release the code and models:
this https URL

我们的研究旨在提高 Transformer 的三维推理能力，在多视图三维人体姿势估计中。通过引入一种新的混合模型，MVGFormer，该模型采用一系列的几何和外观模块，以迭代方式组织。其中，几何模块通过几何方式处理依赖于视角的三维任务，显著改善了模型的泛化能力。外观模块是可学习的，并且专门用于从图像信号中端到端地估计二维姿势，即使在出现遮挡时也能获得准确的估计结果，从而实现了准确性和适用于新的摄像机和几何形状的模型。经过领域内外的评估，我们的模型始终优于最先进的方法，特别是在领域外的情况下有明显的优势。我们将发布代码和模型：this URL

多视角几何变换器用于 3D 人体姿态估计

Multiple View Geometry Transformers for 3D Human Pose Estimation

We present two novel solutions for multi-view 3D human pose estimation based
on new learnable triangulation methods that combine 3D information from
multiple 2D views. The first (baseline) solution is a basic differentiable
algebraic triangulation with an addition of confidence weights estimated from
the input images. The second solution is based on a novel method of volumetric
aggregation from intermediate 2D backbone feature maps. The aggregated volume
is then refined via 3D convolutions that produce final 3D joint heatmaps and
allow modelling a human pose prior. Crucially, both approaches are end-to-end
differentiable, which allows us to directly optimize the target metric. We
demonstrate transferability of the solutions across datasets and considerably
improve the multi-view state of the art on the Human3.6M dataset. Video
demonstration, annotations and additional materials will be posted on our
project page (this https URL).

基于可学习的三角测量方法，使用两种新颖方案对多视图 3D 人体姿势估计进行了探索，其中一种使用置信度权重，另一种是基于体积聚合和三维卷积技术，具有端到端可微的特性，提高了 Human3.6M 数据集上的多视图姿态估计表现。