In this paper, we study the problem of temporal video grounding (TVG), which
aims to predict the starting/ending time points of moments described by a text
sentence within a long untrimmed video. Benefiting from fine-grained 3D visual
features, the TVG techniques have achieved remarkable progress in recent years.
However, the high complexity of 3D convolutional neural networks (CNNs) makes
extracting dense 3D visual features time-consuming, which calls for intensive
memory and computing resources. Towards efficient TVG, we propose a novel
text-visual prompting (TVP) framework, which incorporates optimized
perturbation patterns (that we call 'prompts') into both visual inputs and
textual features of a TVG model. In sharp contrast to 3D CNNs, we show that TVP
allows us to effectively co-train vision encoder and language encoder in a 2D
TVG model and improves the performance of crossmodal feature fusion using only
low-complexity sparse 2D visual features. Further, we propose a
Temporal-Distance IoU (TDIoU) loss for efficient learning of TVG. Experiments
on two benchmark datasets, Charades-STA and ActivityNet Captions datasets,
empirically show that the proposed TVP significantly boosts the performance of
2D TVG (e.g., 9.79% improvement on Charades-STA and 30.77% improvement on
ActivityNet Captions) and achieves 5x inference acceleration over TVG using 3D
visual features. Codes are available at Open.Intel.

本文研究了时空视频定位（TVG）的问题，提出了一种新的文本 - 视觉提示框架（TVP）来优化视频编码器和语言编码器，有效地加强交叉模态特征融合并提高了 TVG 的表现。