Weakly supervised temporal action localization (WS-TAL) is a challenging task
that aims to localize action instances in the given video with video-level
categorical supervision. Both appearance and motion features are used in
previous works, while they do not utilize them in a proper way but apply simple
concatenation or score-level fusion. In this work, we argue that the features
extracted from the pretrained extractor, e.g., I3D, are not the
WS-TALtask-specific features, thus the feature re-calibration is needed for
reducing the task-irrelevant information redundancy. Therefore, we propose a
cross-modal consensus network (CO2-Net) to tackle this problem. In CO2-Net, we
mainly introduce two identical proposed cross-modal consensus modules (CCM)
that design a cross-modal attention mechanism to filter out the task-irrelevant
information redundancy using the global information from the main modality and
the cross-modal local information of the auxiliary modality. Moreover, we treat
the attention weights derived from each CCMas the pseudo targets of the
attention weights derived from another CCM to maintain the consistency between
the predictions derived from two CCMs, forming a mutual learning manner.
Finally, we conduct extensive experiments on two common used temporal action
localization datasets, THUMOS14 and ActivityNet1.2, to verify our method and
achieve the state-of-the-art results. The experimental results show that our
proposed cross-modal consensus module can produce more representative features
for temporal action localization.

本文提出了一种跨模态共识网络以解决弱监督时间动作定位任务中现有特征提取器提取的不是任务特定特征的问题，并使用了跨模态注意机制和相互学习来产生更有代表性的特征。在 THUMOS14 和 ActivityNet1.2 数据集上的实验结果表明，此方法产生的特征实现了时间动作定位的最新成果。