This work aims at advancing temporal action detection (TAD) using an
encoder-decoder framework with action queries, similar to DETR, which has shown
great success in object detection. However, the framework suffers from several
problems if directly applied to TAD: the insufficient exploration of
inter-query relation in the decoder, the inadequate classification training due
to a limited number of training samples, and the unreliable classification
scores at inference. To this end, we first propose a relational attention
mechanism in the decoder, which guides the attention among queries based on
their relations. Moreover, we propose two losses to facilitate and stabilize
the training of action classification. Lastly, we propose to predict the
localization quality of each action query at inference in order to distinguish
high-quality queries. The proposed method, named ReAct, achieves the
state-of-the-art performance on THUMOS14, with much lower computational costs
than previous methods. Besides, extensive ablation studies are conducted to
verify the effectiveness of each proposed component. The code is available at
this https URL.

本文提出了一种名为 ReAct 的时域动作检测模型，采用编解码框架和动作查询相似于 DETR，使用关系注意力机制和两个损失函数加强动作分类的训练，最后通过预测每个动作查询的本地化质量来区分高质量的查询，该模型在 THUMOS14 上取得了最先进的性能，并降低了计算成本。