Multi-person 3D mesh recovery from videos is a critical first step towards
automatic perception of group behavior in virtual reality, physical therapy and
beyond. However, existing approaches rely on multi-stage paradigms, where the
person detection and tracking stages are performed in a multi-person setting,
while temporal dynamics are only modeled for one person at a time.
Consequently, their performance is severely limited by the lack of inter-person
interactions in the spatial-temporal mesh recovery, as well as by detection and
tracking defects. To address these challenges, we propose the Coordinate
transFormer (CoordFormer) that directly models multi-person spatial-temporal
relations and simultaneously performs multi-mesh recovery in an end-to-end
manner. Instead of partitioning the feature map into coarse-scale patch-wise
tokens, CoordFormer leverages a novel Coordinate-Aware Attention to preserve
pixel-level spatial-temporal coordinate information. Additionally, we propose a
simple, yet effective Body Center Attention mechanism to fuse position
information. Extensive experiments on the 3DPW dataset demonstrate that
CoordFormer significantly improves the state-of-the-art, outperforming the
previously best results by 4.2%, 8.8% and 4.7% according to the MPJPE, PAMPJPE,
and PVE metrics, respectively, while being 40% faster than recent video-based
approaches. The released code can be found at
this https URL

多人三维网格恢复是实现虚拟现实、物理治疗等领域团体行为自动感知的关键步骤。本研究提出了 CoordFormer 模型，通过建模多人空时关系并同时进行多网格恢复，显著提高了最新成果，超过了 MPJPE、PAMPJPE 和 PVE 指标，速度比最近的基于视频的方法快 40%。