Temporal language localization in videos aims to ground one video segment in
an untrimmed video based on a given sentence query. To tackle this task,
designing an effective model to extract ground-ing information from both visual
and textual modalities is crucial. However, most previous attempts in this
field only focus on unidirectional interactions from video to query, which
emphasizes which words to listen and attends to sentence information via
vanilla soft attention, but clues from query-by-video interactions implying
where to look are not taken into consideration. In this paper, we propose a
Fine-grained Iterative Attention Network (FIAN) that consists of an iterative
attention module for bilateral query-video in-formation extraction.
Specifically, in the iterative attention module, each word in the query is
first enhanced by attending to each frame in the video through fine-grained
attention, then video iteratively attends to the integrated query. Finally,
both video and query information is utilized to provide robust cross-modal
representation for further moment localization. In addition, to better predict
the target segment, we propose a content-oriented localization strategy instead
of applying recent anchor-based localization. We evaluate the proposed method
on three challenging public benchmarks: Ac-tivityNet Captions, TACoS, and
Charades-STA. FIAN significantly outperforms the state-of-the-art approaches.

本文提出了一种基于 Fine-grained Iterative Attention Network (FIAN) 的跨模态时间语言定位方法，该方法采用内容定向定位策略而非现有的基于 Anchor 的方法，并在多个公开基准测试中表现出优异的性能。