Temporal action detection aims to predict the time intervals and the classes
of action instances in the video. Despite the promising performance, existing
two-stream models exhibit slow inference speed due to their reliance on
computationally expensive optical flow. In this paper, we introduce a
decomposed cross-modal distillation framework to build a strong RGB-based
detector by transferring knowledge of the motion modality. Specifically,
instead of direct distillation, we propose to separately learn RGB and motion
representations, which are in turn combined to perform action localization. The
dual-branch design and the asymmetric training objectives enable effective
motion knowledge transfer while preserving RGB information intact. In addition,
we introduce a local attentive fusion to better exploit the multimodal
complementarity. It is designed to preserve the local discriminability of the
features that is important for action localization. Extensive experiments on
the benchmarks verify the effectiveness of the proposed method in enhancing
RGB-based action detectors. Notably, our framework is agnostic to backbones and
detection heads, bringing consistent gains across different model combinations.

本文介绍了一种基于交叉模态蒸馏的分解式跨空间学习框架，通过学习 RGB 和运动表征并结合两者来完成动作定位，极大地提高了基于 RGB 的动作检测器的性能，该框架不受骨干网络和检测头的影响。