Current video summarization methods primarily depend on supervised computer
vision techniques, which demands time-consuming manual annotations. Further,
the annotations are always subjective which make this task more challenging. To
address these issues, we analyzed the feasibility in transforming the video
summarization into a text summary task and leverage Large Language Models
(LLMs) to boost video summarization. This paper proposes a novel
self-supervised framework for video summarization guided by LLMs. Our method
begins by generating captions for video frames, which are then synthesized into
text summaries by LLMs. Subsequently, we measure semantic distance between the
frame captions and the text summary. It's worth noting that we propose a novel
loss function to optimize our model according to the diversity of the video.
Finally, the summarized video can be generated by selecting the frames whose
captions are similar with the text summary. Our model achieves competitive
results against other state-of-the-art methods and paves a novel pathway in
video summarization.

通过使用大型语言模型作为引导，本文提出了一种新颖的自监督视频摘要框架，通过生成视频帧的字幕，并将其合成为文本摘要，衡量帧字幕和文本摘要之间的语义距离，最终选择与文本摘要相似的帧来生成摘要视频。该方法在视频摘要任务中取得了有竞争力的结果，并为视频摘要领域开辟了新的道路。