A key solution to temporal sentence grounding (TSG) exists in how to learn
effective alignment between vision and language features extracted from an
untrimmed video and a sentence description. Existing methods mainly leverage
vanilla soft attention to perform the alignment in a single-step process.
However, such single-step attention is insufficient in practice, since
complicated relations between inter- and intra-modality are usually obtained
through multi-step reasoning. In this paper, we propose an Iterative Alignment
Network (IA-Net) for TSG task, which iteratively interacts inter- and
intra-modal features within multiple steps for more accurate grounding.
Specifically, during the iterative reasoning process, we pad multi-modal
features with learnable parameters to alleviate the nowhere-to-attend problem
of non-matched frame-word pairs, and enhance the basic co-attention mechanism
in a parallel manner. To further calibrate the misaligned attention caused by
each reasoning step, we also devise a calibration module following each
attention module to refine the alignment knowledge. With such iterative
alignment scheme, our IA-Net can robustly capture the fine-grained relations
between vision and language domains step-by-step for progressively reasoning
the temporal boundaries. Extensive experiments conducted on three challenging
benchmarks demonstrate that our proposed model performs better than the
state-of-the-arts.

本文提出了一个迭代对齐网络 (IA-Net) 来解决 TSG 任务，通过多步推理来互动地对齐视觉和语言特征，通过学习参数对多模态特征进行填充来缓解非匹配的问题，并且在并行方式下增强基本的共同注意力机制。为了进一步校准由每个推理步骤引起的不匹配的注意力，我们还设计了一个校准模块。本文的 IA-Net 模型在三个具有挑战性的基准测试中表现出更好的性能。