Some group activities, such as team sports and choreographed dances, involve
closely coupled interaction between participants. Here we investigate the tasks
of inferring and predicting participant behavior, in terms of motion paths and
actions, under such conditions. We narrow the problem to that of estimating how
a set target participants react to the behavior of other observed participants.
Our key idea is to model the spatio-temporal relations among participants in a
manner that is robust to error accumulation during frame-wise inference and
prediction. We propose a novel Entry-Flipped Transformer (EF-Transformer),
which models the relations of participants by attention mechanisms on both
spatial and temporal domains. Unlike typical transformers, we tackle the
problem of error accumulation by flipping the order of query, key, and value
entries, to increase the importance and fidelity of observed features in the
current frame. Comparative experiments show that our EF-Transformer achieves
the best performance on a newly-collected tennis doubles dataset, a Ceilidh
dance dataset, and two pedestrian datasets. Furthermore, it is also
demonstrated that our EF-Transformer is better at limiting accumulated errors
and recovering from wrong estimations.

本文提出了一种新颖的 EF-Transformer 模型，该模型通过对空间和时间域的注意机制，建模参与者之间的关系，从而更好地推断和预测参与者的行为，实验结果表明，相比其他方法，EF-Transformer 在团体活动中的表现更好，能够更好地应对累计误差和错误估计的问题。