We tackle the task of video moment retrieval (VMR), which aims to localize a
specific moment in a video according to a textual query. Existing methods
primarily model the matching relationship between query and moment by complex
cross-modal interactions. Despite their effectiveness, current models mostly
exploit dataset biases while ignoring the video content, thus leading to poor
generalizability. We argue that the issue is caused by the hidden confounder in
VMR, {i.e., temporal location of moments}, that spuriously correlates the model
input and prediction. How to design robust matching models against the temporal
location biases is crucial but, as far as we know, has not been studied yet for
VMR.
To fill the research gap, we propose a causality-inspired VMR framework that
builds structural causal model to capture the true effect of query and video
content on the prediction. Specifically, we develop a Deconfounded Cross-modal
Matching (DCM) method to remove the confounding effects of moment location. It
first disentangles moment representation to infer the core feature of visual
content, and then applies causal intervention on the disentangled multimodal
input based on backdoor adjustment, which forces the model to fairly
incorporate each possible location of the target into consideration. Extensive
experiments clearly show that our approach can achieve significant improvement
over the state-of-the-art methods in terms of both accuracy and generalization
(Codes:
\color{blue}{https://github.com/Xun-Yang/Causal_Video_Moment_Retrieval}

该研究提出了一种基于因果关系的视频时刻检索框架，利用 Deconfounded Cross-modal Matching（DCM）方法去除时刻位置的混淆效应，并在考虑所有可能的目标位置的情况下，公平地将查询和视频内容纳入模型中以提高准确性和泛化性能。