We address the problem of retrieving a specific moment from an untrimmed
video by a query sentence. This is a challenging problem because a target
moment may take place in relations to other temporal moments in the untrimmed
video. Existing methods cannot tackle this challenge well since they consider
temporal moments individually and neglect the temporal dependencies. In this
paper, we model the temporal relations between video moments by a
two-dimensional map, where one dimension indicates the starting time of a
moment and the other indicates the end time. This 2D temporal map can cover
diverse video moments with different lengths, while representing their adjacent
relations. Based on the 2D map, we propose a Temporal Adjacent Network
(2D-TAN), a single-shot framework for moment localization. It is capable of
encoding the adjacent temporal relation, while learning discriminative features
for matching video moments with referring expressions. We evaluate the proposed
2D-TAN on three challenging benchmarks, i.e., Charades-STA, ActivityNet
Captions, and TACoS, where our 2D-TAN outperforms the state-of-the-art.

本文提出了一个基于 2D 时间映射的 Temporal Adjacent Network（2D-TAN）框架，用于在未修剪的视频中匹配与查询句子相关的特定时间点，该框架可以编码相邻的时间关系，并学习具有区分性的特征以提高活动识别的准确性，通过在 Charades-STA，ActivityNet Captions 和 TACoS 等数据集上的表现，证明了该方法的优越性。