Video Moment Retrieval (VMR) aims to localize a specific temporal segment
within an untrimmed long video given a natural language query. Existing methods
often suffer from inadequate training annotations, i.e., the sentence typically
matches with a fraction of the prominent video content in the foreground with
limited wording diversity. This intrinsic modality imbalance leaves a
considerable portion of visual information remaining unaligned with text. It
confines the cross-modal alignment knowledge within the scope of a limited text
corpus, thereby leading to sub-optimal visual-textual modeling and poor
generalizability. By leveraging the visual-textual understanding capability of
multi-modal large language models (MLLM), in this work, we take an MLLM as a
video narrator to generate plausible textual descriptions of the video, thereby
mitigating the modality imbalance and boosting the temporal localization. To
effectively maintain temporal sensibility for localization, we design to get
text narratives for each certain video timestamp and construct a structured
text paragraph with time information, which is temporally aligned with the
visual content. Then we perform cross-modal feature merging between the
temporal-aware narratives and corresponding video temporal features to produce
semantic-enhanced video representation sequences for query localization.
Subsequently, we introduce a uni-modal narrative-query matching mechanism,
which encourages the model to extract complementary information from contextual
cohesive descriptions for improved retrieval. Extensive experiments on two
benchmarks show the effectiveness and generalizability of our proposed method.

利用多模态大型语言模型 (MLLM) 的视觉文本理解能力，本研究以 MLLM 作为视频的叙述者，生成视频的文本描述，从而减少模态不平衡并提高时间定位的准确性。通过获取视频每个时间戳的文本叙述并构建带有时间信息的结构化文本段落，与视觉内容进行时间对齐。然后，将时间感知的叙述和相应的视频时间特征进行跨模态特征融合，生成语义增强的视频表示序列用于查询定位。接下来，引入了一种单模态的叙述 - 查询匹配机制，鼓励模型从上下文连贯的描述中提取互补信息，以改善检索效果。该方法在两个基准测试上进行了广泛实验，证明了其有效性和普适性。