Automatic movie narration targets at creating video-aligned plot descriptions
to assist visually impaired audiences. It differs from standard video
captioning in that it requires not only describing key visual details but also
inferring the plots developed across multiple movie shots, thus posing unique
and ongoing challenges. To advance the development of automatic movie narrating
systems, we first revisit the limitations of existing datasets and develop a
large-scale, bilingual movie narration dataset, Movie101v2. Second, taking into
account the essential difficulties in achieving applicable movie narration, we
break the long-term goal into three progressive stages and tentatively focus on
the initial stages featuring understanding within individual clips. We also
introduce a new narration assessment to align with our staged task goals.
Third, using our new dataset, we baseline several leading large vision-language
models, including GPT-4V, and conduct in-depth investigations into the
challenges current models face for movie narration generation. Our findings
reveal that achieving applicable movie narration generation is a fascinating
goal that requires thorough research.

通过创建与视频对齐的情节描述来辅助视觉障碍观众，自动电影叙述与标准视频字幕不同，它需要描述关键的视觉细节，同时推断跨多个电影镜头发展的情节，因此存在独特和持续的挑战。为了推进自动电影叙述系统的发展，我们首先重新审视现有数据集的限制，并开发了一个大规模的双语电影叙述数据集 Movie101v2。其次，考虑到实现可应用的电影叙述的基本困难，我们将长期目标分为三个渐进阶段，并临时聚焦于特定片段内的理解。我们还引入了一种新的叙述评估来与我们的阶段性任务目标对齐。第三，利用我们的新数据集，我们对几种主要的大规模视觉语言模型进行了基准测试，包括 GPT-4V，并对当前模型在电影叙述生成方面面临的挑战进行了深入研究。我们的研究结果表明，实现可应用的电影叙述生成是一个引人入胜的目标，需要深入研究。