Estimating the 2D human poses in each view is typically the first step in
calibrated multi-view 3D pose estimation. But the performance of 2D pose
detectors suffers from challenging situations such as occlusions and oblique
viewing angles. To address these challenges, previous works derive
point-to-point correspondences between different views from epipolar geometry
and utilize the correspondences to merge prediction heatmaps or feature
representations. Instead of post-prediction merge/calibration, here we
introduce a transformer framework for multi-view 3D pose estimation, aiming at
directly improving individual 2D predictors by integrating information from
different views. Inspired by previous multi-modal transformers, we design a
unified transformer architecture, named TransFusion, to fuse cues from both
current views and neighboring views. Moreover, we propose the concept of
epipolar field to encode 3D positional information into the transformer model.
The 3D position encoding guided by the epipolar field provides an efficient way
of encoding correspondences between pixels of different views. Experiments on
Human 3.6M and Ski-Pose show that our method is more efficient and has
consistent improvements compared to other fusion methods. Specifically, we
achieve 25.8 mm MPJPE on Human 3.6M with only 5M parameters on 256 x 256
resolution.

该研究论文提出了一种可应用于多视角 3D 姿态估计中的变换器框架，该框架可以直接整合来自不同视角的信息，以改善 2D 预测器的性能。同时，研究人员还提出了一个名为 Epipolar field 的概念，它可以将 3D 位置信息编码到变换器模型中。实验证明，该方法更加高效，与其他融合方法相比有一致的改进。