The task of retrieving clips within videos based on a given natural language
query requires cross-modal reasoning over multiple frames. Prior approaches
such as sliding window classifiers are inefficient, while text-clip similarity
driven ranking-based approaches such as segment proposal networks are far more
complicated. In order to select the most relevant video clip corresponding to
the given text description, we propose a novel extractive approach that
predicts the start and end frames by leveraging cross-modal interactions
between the text and video - this removes the need to retrieve and re-rank
multiple proposal segments. Using recurrent networks we encode the two
modalities into a joint representation which is then used in different variants
of start-end frame predictor networks. Through extensive experimentation and
ablative analysis, we demonstrate that our simple and elegant approach
significantly outperforms state of the art on two datasets and has comparable
performance on a third.

该文章提出了一种新的、能够利用文本和视频间的交互，预测起始和结束帧的提取方法，以避免检索和重新排序多个候选段落。使用递归网络将两种模态编码成共同表示，该方法在多项实验和去除分析中表现显著，比现有技术提升了性能。