Zero-shot Natural Language-Video Localization (NLVL) methods have exhibited
promising results in training NLVL models exclusively with raw video data by
dynamically generating video segments and pseudo-query annotations. However,
existing pseudo-queries often lack grounding in the source video, resulting in
unstructured and disjointed content. In this paper, we investigate the
effectiveness of commonsense reasoning in zero-shot NLVL. Specifically, we
present CORONET, a zero-shot NLVL framework that leverages commonsense to
bridge the gap between videos and generated pseudo-queries via a commonsense
enhancement module. CORONET employs Graph Convolution Networks (GCN) to encode
commonsense information extracted from a knowledge graph, conditioned on the
video, and cross-attention mechanisms to enhance the encoded video and
pseudo-query representations prior to localization. Through empirical
evaluations on two benchmark datasets, we demonstrate that CORONET surpasses
both zero-shot and weakly supervised baselines, achieving improvements up to
32.13% across various recall thresholds and up to 6.33% in mIoU. These results
underscore the significance of leveraging commonsense reasoning for zero-shot
NLVL.

本文研究了零样本自然语言视频定位（NLVL）中常识推理的有效性，并提出了一个名为 CORONET 的零样本 NLVL 框架，利用常识知识图谱，通过常识增强模块来弥合视频和生成的伪查询之间的差距。通过对两个基准数据集的实证评估，证明 CORONET 超过了零样本和弱监督的基准，不同召回阈值下的改进幅度高达 32.13％，在 mIoU 方面的改进幅度高达 6.33％，突出了利用常识推理进行零样本 NLVL 的重要性。