The majority of traditional text-to-video retrieval systems operate in static
environments, i.e., there is no interaction between the user and the agent
beyond the initial textual query provided by the user. This can be sub-optimal
if the initial query has ambiguities, which would lead to many falsely
retrieved videos. To overcome this limitation, we propose a novel framework for
Video Retrieval using Dialog (ViReD), which enables the user to interact with
an AI agent via multiple rounds of dialog, where the user refines retrieved
results by answering questions generated by an AI agent. Our novel multimodal
question generator learns to ask questions that maximize the subsequent video
retrieval performance using (i) the video candidates retrieved during the last
round of interaction with the user and (ii) the text-based dialog history
documenting all previous interactions, to generate questions that incorporate
both visual and linguistic cues relevant to video retrieval. Furthermore, to
generate maximally informative questions, we propose an Information-Guided
Supervision (IGS), which guides the question generator to ask questions that
would boost subsequent video retrieval accuracy. We validate the effectiveness
of our interactive ViReD framework on the AVSD dataset, showing that our
interactive method performs significantly better than traditional
non-interactive video retrieval systems. We also demonstrate that our proposed
approach generalizes to the real-world settings that involve interactions with
real humans, thus, demonstrating the robustness and generality of our framework

提出了一种具备多轮对话交互能力的视频检索框架，该框架包括 AI agent、多模态问答生成器及信息指导监督器，实验表明其效果显著优于传统非交互方式的视频检索系统。