Video Moment Retrieval (VMR) aims to retrieve temporal segments in untrimmed
videos corresponding to a given language query by constructing cross-modal
alignment strategies. However, these existing strategies are often sub-optimal
since they ignore the modality imbalance problem, \textit{i.e.}, the semantic
richness inherent in videos far exceeds that of a given limited-length
sentence. Therefore, in pursuit of better alignment, a natural idea is
enhancing the video modality to filter out query-irrelevant semantics, and
enhancing the text modality to capture more segment-relevant knowledge. In this
paper, we introduce Modal-Enhanced Semantic Modeling (MESM), a novel framework
for more balanced alignment through enhancing features at two levels. First, we
enhance the video modality at the frame-word level through word reconstruction.
This strategy emphasizes the portions associated with query words in
frame-level features while suppressing irrelevant parts. Therefore, the
enhanced video contains less redundant semantics and is more balanced with the
textual modality. Second, we enhance the textual modality at the
segment-sentence level by learning complementary knowledge from context
sentences and ground-truth segments. With the knowledge added to the query, the
textual modality thus maintains more meaningful semantics and is more balanced
with the video modality. By implementing two levels of MESM, the semantic
information from both modalities is more balanced to align, thereby bridging
the modality gap. Experiments on three widely used benchmarks, including the
out-of-distribution settings, show that the proposed framework achieves a new
start-of-the-art performance with notable generalization ability (e.g., 4.42%
and 7.69% average gains of R1@0.7 on Charades-STA and Charades-CG). The code
will be available at this https URL

通过提升视频模态和文本模态的特征，Modal-Enhanced Semantic Modeling（MESM）框架在视频短片检索中实现了更平衡的对齐，填补了形式上不平衡的模态差距。实验证明该框架在多个基准测试上取得了显著的泛化能力和最佳效果。