Video-Text Pre-training (VTP) aims to learn transferable representations for
various downstream tasks from large-scale web videos. To date, almost all
existing VTP methods are limited to retrieval-based downstream tasks, e.g.,
video retrieval, whereas their transfer potentials on localization-based tasks,
e.g., temporal grounding, are under-explored. In this paper, we experimentally
analyze and demonstrate the incompatibility of current VTP methods with
localization tasks, and propose a novel Localization-oriented Video-Text
Pre-training framework, dubbed as LocVTP. Specifically, we perform the
fine-grained contrastive alignment as a complement to the coarse-grained one by
a clip-word correspondence discovery scheme. To further enhance the temporal
reasoning ability of the learned feature, we propose a context projection head
and a temporal aware contrastive loss to perceive the contextual relationships.
Extensive experiments on four downstream tasks across six datasets demonstrate
that our LocVTP achieves state-of-the-art performance on both retrieval-based
and localization-based tasks. Furthermore, we conduct comprehensive ablation
studies and thorough analyses to explore the optimum model designs and training
strategies.

本文提出了一种面向本地化任务的视频文本预训练框架 LocVTP，通过精细对比对齐和上下文感知等机制，提高了其学到的特征的时空推理能力和传递性，实现了在四个下游任务上的最优表现。