Video captioning is a challenging task as it needs to accurately transform visual understanding into natural language description. To date, state-of-the-art methods inadequately model global-local representation across video frames for caption generation, leaving plenty of room for improvement. In this work, we approach the video captioning task from a new perspective and propose a GL-RG framework for video captioning, namely a \textbf{G}lobal-\textbf{L}ocal \textbf{R}epresentation \textbf{G}ranularity. Our GL-RG demonstrates three advantages over the prior efforts: 1) we explicitly exploit extensive visual representations from different video ranges to improve linguistic expression; 2) we devise a novel global-local encoder to produce rich semantic vocabulary to obtain a descriptive granularity of video contents across frames; 3) we develop an incremental training strategy which organizes model learning in an incremental fashion to incur an optimal captioning behavior. Experimental results on the challenging MSR-VTT and MSVD datasets show that our DL-RG outperforms recent state-of-the-art methods by a significant margin. Code is available at \url{https://github.com/ylqi/GL-RG}.

本文提出了一种全局-局部表示粒度框架（GL-RG）来生成视频字幕，通过显式利用来自不同视频范围的丰富视觉表示，设计了一个新颖的全局-局部编码器来产生丰富的语义词汇，以获得帧间视频内容的描述性粒度，采用增量训练策略组织模型学习，实验结果表明GL-RG在MSR-VTT和MSVD数据集上的性能显著优于最近的最先进方法。

GL-RG: 视频字幕生成中的全局与局部表示层级