Recently, researchers have attempted to investigate the capability of LLMs in
handling videos and proposed several video LLM models. However, the ability of
LLMs to handle video grounding (VG), which is an important time-related video
task requiring the model to precisely locate the start and end timestamps of
temporal moments in videos that match the given textual queries, still remains
unclear and unexplored in literature. To fill the gap, in this paper, we
propose the LLM4VG benchmark, which systematically evaluates the performance of
different LLMs on video grounding tasks. Based on our proposed LLM4VG, we
design extensive experiments to examine two groups of video LLM models on video
grounding: (i) the video LLMs trained on the text-video pairs (denoted as
VidLLM), and (ii) the LLMs combined with pretrained visual description models
such as the video/image captioning model. We propose prompt methods to
integrate the instruction of VG and description from different kinds of
generators, including caption-based generators for direct visual description
and VQA-based generators for information enhancement. We also provide
comprehensive comparisons of various VidLLMs and explore the influence of
different choices of visual models, LLMs, prompt designs, etc, as well. Our
experimental evaluations lead to two conclusions: (i) the existing VidLLMs are
still far away from achieving satisfactory video grounding performance, and
more time-related video tasks should be included to further fine-tune these
models, and (ii) the combination of LLMs and visual models shows preliminary
abilities for video grounding with considerable potential for improvement by
resorting to more reliable models and further guidance of prompt instructions.

近年来，研究人员试图调查 LLM 在处理视频方面的能力，并提出了几种视频 LLM 模型。然而，LLM 在处理视频对齐（VG）方面的能力仍然不清楚，也没有在文献中进行探索。为了填补这一空白，本文提出了 LLM4VG 基准测试，对不同的 LLM 在视频对齐任务上的性能进行系统评估。基于我们提出的 LLM4VG，我们设计了大量实验，检查了两组视频 LLM 模型在视频对齐上的表现：（i）基于文本 - 视频对训练的视频 LLM（标记为 VidLLM），以及（ii）与预训练的视觉描述模型（如视频 / 图像字幕模型）结合的 LLM。我们提出了整合 VG 指令和来自不同类型生成器的描述的方法，包括用于直接视觉描述的基于字幕的生成器和用于信息增强的基于 VQA 的生成器。我们还对各种 VidLLM 进行了全面比较，并探讨了不同视觉模型、LLM、提示设计等的影响。我们的实验评估得出了两个结论：（i）现有的 VidLLM 离实现令人满意的视频对齐性能还有很长的路要走，需要进一步微调这些模型以包含更多的与时间相关的视频任务；（ii）LLM 和视觉模型的组合显示出初步的视频对齐能力，通过采用更可靠的模型和进一步的提示指导，这种能力具有可观的改进潜力。