The goal of visual answering localization (VAL) in the video is to obtain a
relevant and concise time clip from a video as the answer to the given natural
language question. Early methods are based on the interaction modelling between
video and text to predict the visual answer by the visual predictor. Later,
using the textual predictor with subtitles for the VAL proves to be more
precise. However, these existing methods still have cross-modal knowledge
deviations from visual frames or textual subtitles. In this paper, we propose a
cross-modal mutual knowledge transfer span localization (MutualSL) method to
reduce the knowledge deviation. MutualSL has both visual predictor and textual
predictor, where we expect the prediction results of these both to be
consistent, so as to promote semantic knowledge understanding between
cross-modalities. On this basis, we design a one-way dynamic loss function to
dynamically adjust the proportion of knowledge transfer. We have conducted
extensive experiments on three public datasets for evaluation. The experimental
results show that our method outperforms other competitive state-of-the-art
(SOTA) methods, demonstrating its effectiveness.

本文提出了一种跨模态互相知识传递的跨模态互知跨度本地化方法 (MutualSL)，它包含视觉预测器和文本预测器两个部分，旨在通过构建一种双向动态损失函数以调整传递比例来提高语义知识的理解，以获取视频中自然语言问题的相关和简明时间剪辑答案。实验结果表明，该方法优于其他竞争同类方法，展示了其有效性。