We present COMEDIAN, a novel pipeline to initialize spatio-temporal transformers for action spotting, which involves self-supervised learning and knowledge distillation. Action spotting is a timestamp-level temporal action detection task. Our pipeline consists of three steps, with two initialization stages. First, we perform self-supervised initialization of a spatial transformer using short videos as input. Additionally, we initialize a temporal transformer that enhances the spatial transformer's outputs with global context through knowledge distillation from a pre-computed feature bank aligned with each short video segment. In the final step, we fine-tune the transformers to the action spotting task. The experiments, conducted on the SoccerNet-v2 dataset, demonstrate state-of-the-art performance and validate the effectiveness of COMEDIAN's pretraining paradigm. Our results highlight several advantages of our pretraining pipeline, including improved performance and faster convergence compared to non-pretrained models.

我们提出了COMEDIAN，这是一种新颖的用于动作检测的时空变换器初始化流程，其中包括自监督学习和知识蒸馏。通过三个步骤和两个初始化阶段，我们的流程实现了时间戳级别的动作检测任务。实验表明，我们的预训练模式具有卓越的性能和快速收敛，相比未经预训练的模型，我们的预训练流程具有改进的性能和更快的收敛速度。

COMEDIAN：自我监督学习与知识蒸馏在使用Transformers进行动作定位中的应用