For a complete comprehension of multi-person scenes, it is essential to go
beyond basic tasks like detection and tracking. Higher-level tasks, such as
understanding the interactions and social activities among individuals, are
also crucial. Progress towards models that can fully understand scenes
involving multiple people is hindered by a lack of sufficient annotated data
for such high-level tasks. To address this challenge, we introduce Social-MAE,
a simple yet effective transformer-based masked autoencoder framework for
multi-person human motion data. The framework uses masked modeling to pre-train
the encoder to reconstruct masked human joint trajectories, enabling it to
learn generalizable and data efficient representations of motion in human
crowded scenes. Social-MAE comprises a transformer as the MAE encoder and a
lighter-weight transformer as the MAE decoder which operates on multi-person
joints' trajectory in the frequency domain. After the reconstruction task, the
MAE decoder is replaced with a task-specific decoder and the model is
fine-tuned end-to-end for a variety of high-level social tasks. Our proposed
model combined with our pre-training approach achieves the state-of-the-art
results on various high-level social tasks, including multi-person pose
forecasting, social grouping, and social action understanding. These
improvements are demonstrated across four popular multi-person datasets
encompassing both human 2D and 3D body pose.

为了理解多人场景，我们引入了 Social-MAE，一种基于 transformer 的带掩码自编码器框架，用于多人人体运动数据，并在高级社交任务中达到了最先进的结果。