Some group activities, such as team sports and choreographed dances, involve
closely coupled interaction between participants. Here we investigate the tasks
of inferring and predicting participant behavior, in terms of motion paths and
actions, under such conditions. We narrow the problem to that of estimating how
a set target participants react to the behavior of other observed participants.
Our key idea is to model the spatio-temporal relations among participants in a
manner that is robust to error accumulation during frame-wise inference and
prediction. We propose a novel Entry-Flipped Transformer (EF-Transformer),
which models the relations of participants by attention mechanisms on both
spatial and temporal domains. Unlike typical transformers, we tackle the
problem of error accumulation by flipping the order of query, key, and value
entries, to increase the importance and fidelity of observed features in the
current frame. Comparative experiments show that our EF-Transformer achieves
the best performance on a newly-collected tennis doubles dataset, a Ceilidh
dance dataset, and two pedestrian datasets. Furthermore, it is also
demonstrated that our EF-Transformer is better at limiting accumulated errors
and recovering from wrong estimations.

本文提出了一种新颖的 EF-Transformer 模型，该模型通过对空间和时间域的注意机制，建模参与者之间的关系，从而更好地推断和预测参与者的行为，实验结果表明，相比其他方法，EF-Transformer 在团体活动中的表现更好，能够更好地应对累计误差和错误估计的问题。

逆序输入变换器用于参与者行为推理和预测

Entry-Flipped Transformer for Inference and Prediction of Participant  Behavior

Extending state-of-the-art object detectors from image to video is
challenging. The accuracy of detection suffers from degenerated object
appearances in videos, e.g., motion blur, video defocus, rare poses, etc.
Existing work attempts to exploit temporal information on box level, but such
methods are not trained end-to-end. We present flow-guided feature aggregation,
an accurate and end-to-end learning framework for video object detection. It
leverages temporal coherence on feature level instead. It improves the
per-frame features by aggregation of nearby features along the motion paths,
and thus improves the video recognition accuracy. Our method significantly
improves upon strong single-frame baselines in ImageNet VID, especially for
more challenging fast moving objects. Our framework is principled, and on par
with the best engineered systems winning the ImageNet VID challenges 2016,
without additional bells-and-whistles. The proposed method, together with Deep
Feature Flow, powered the winning entry of ImageNet VID challenges 2017. The
code is available at
this https URL

该研究提出了一种基于流引导特征聚合的视频物体检测学习框架，该框架通过沿着运动路径聚合邻近特征来提高视频的识别精度，避免了视频中物体外观退化的问题，并且完全可训练，与 ImageNet VID 挑战的最佳工程系统相当