Video captioning combines video understanding and language generation. Different from image captioning that describes a static image with details of almost every object, video captioning usually considers a sequence of frames and biases towards focused objects, e.g., the objects that stay in focus regardless of the changing background. Therefore, detecting and properly accommodating focused objects is critical in video captioning. To enforce the description of focused objects and achieve controllable video captioning, we propose an Object-Oriented Non-Autoregressive approach (O2NA), which performs caption generation in three steps: 1) identify the focused objects and predict their locations in the target caption; 2) generate the related attribute words and relation words of these focused objects to form a draft caption; and 3) combine video information to refine the draft caption to a fluent final caption. Since the focused objects are generated and located ahead of other words, it is difficult to apply the word-by-word autoregressive generation process; instead, we adopt a non-autoregressive approach. The experiments on two benchmark datasets, i.e., MSR-VTT and MSVD, demonstrate the effectiveness of O2NA, which achieves results competitive with the state-of-the-arts but with both higher diversity and higher inference speed.

本文介绍了一种基于物体的非自回归方法(O2NA)用于视频字幕生成，它包括确定聚焦对象，生成草案字幕，以及将视频信息与草案字幕结合以生成最终流畅字幕等步骤，实验结果表明O2NA在MSR-VTT和MSVD两个基准数据数据集上具有与现有最先进技术相当的结果，但具有更高的多样性和推理速度。

O2NA：一种用于可控视频字幕生成的面向对象非自回归方法