Video moment retrieval (VMR) aims to identify the specific moment in an
untrimmed video for a given natural language query. However, this task is prone
to suffer the weak visual-textual alignment problem from query ambiguity,
potentially limiting further performance gains and generalization capability.
Due to the complex multimodal interactions in videos, a query may not fully
cover the relevant details of the corresponding moment, and the moment may
contain misaligned and irrelevant frames. To tackle this problem, we propose a
straightforward yet effective model, called Background-aware Moment DEtection
TRansformer (BM-DETR). Given a target query and its moment, BM-DETR also takes
negative queries corresponding to different moments. Specifically, our model
learns to predict the target moment from the joint probability of the given
query and the complement of negative queries for each candidate frame. In this
way, it leverages the surrounding background to consider relative importance,
improving moment sensitivity. Extensive experiments on Charades-STA and
QVHighlights demonstrate the effectiveness of our model. Moreover, we show that
BM-DETR can perform robustly in three challenging VMR scenarios, such as
several out-of-distribution test cases, demonstrating superior generalization
ability.

提出一种名为 BM-DETR 的背景感知时刻检测变压器模型，通过利用负查询和周围的背景来考虑相关性并提高时刻灵敏度，从而提高视频时刻检索（VMR）的效果和泛化能力。