Learning spatial-temporal relation among multiple actors is crucial for group
activity recognition. Different group activities often show the diversified
interactions between actors in the video. Hence, it is often difficult to model
complex group activities from a single view of spatial-temporal actor
evolution. To tackle this problem, we propose a distinct Dual-path Actor
Interaction (DualAI) framework, which flexibly arranges spatial and temporal
transformers in two complementary orders, enhancing actor relations by
integrating merits from different spatiotemporal paths. Moreover, we introduce
a novel Multi-scale Actor Contrastive Loss (MAC-Loss) between two interactive
paths of Dual-AI. Via self-supervised actor consistency in both frame and video
levels, MAC-Loss can effectively distinguish individual actor representations
to reduce action confusion among different actors. Consequently, our Dual-AI
can boost group activity recognition by fusing such discriminative features of
different actors. To evaluate the proposed approach, we conduct extensive
experiments on the widely used benchmarks, including Volleyball, Collective
Activity, and NBA datasets. The proposed Dual-AI achieves state-of-the-art
performance on all these datasets. It is worth noting the proposed Dual-AI with
50% training data outperforms a number of recent approaches with 100% training
data. This confirms the generalization power of Dual-AI for group activity
recognition, even under the challenging scenarios of limited supervision.

提出 Dual-path Actor Interaction (DualAI) 框架和 Multi-scale Actor Contrastive Loss (MAC-Loss) 方法，通过可灵活布置空间和时间转换器、自监督 actor 一致性、二交互路径的 Multi-scale Actor Contrastive Loss (MAC-Loss) 等手段，增强演员之间的关系，并通过融合不同演员的特征来提高群体活动识别的性能。在多个数据集上的实验表明，该方法具有很好的推广能力，在有限监督的情况下仍能取得最先进的性能。