We present Multi-view Pose transformer (MvP) for estimating multi-person 3D
poses from multi-view images. Instead of estimating 3D joint locations from
costly volumetric representation or reconstructing the per-person 3D pose from
multiple detected 2D poses as in previous methods, MvP directly regresses the
multi-person 3D poses in a clean and efficient way, without relying on
intermediate tasks. Specifically, MvP represents skeleton joints as learnable
query embeddings and let them progressively attend to and reason over the
multi-view information from the input images to directly regress the actual 3D
joint locations. To improve the accuracy of such a simple pipeline, MvP
presents a hierarchical scheme to concisely represent query embeddings of
multi-person skeleton joints and introduces an input-dependent query adaptation
approach. Further, MvP designs a novel geometrically guided attention
mechanism, called projective attention, to more precisely fuse the cross-view
information for each joint. MvP also introduces a RayConv operation to
integrate the view-dependent camera geometry into the feature representations
for augmenting the projective attention. We show experimentally that our MvP
model outperforms the state-of-the-art methods on several benchmarks while
being much more efficient. Notably, it achieves 92.3% AP25 on the challenging
Panoptic dataset, improving upon the previous best approach [36] by 9.8%. MvP
is general and also extendable to recovering human mesh represented by the SMPL
model, thus useful for modeling multi-person body shapes. Code and models are
available at this https URL

该研究提出了一种名为 MvP 的多视角姿态变换器，用于从多视角图像中估计多人的三维姿态，该方法可以直接回归多人三维姿态，并使用几种技术来提高准确性，该模型在多项指标上表现优秀。