Video captioning is a challenging task that requires a deep understanding of
visual scenes. State-of-the-art methods generate captions using either
scene-level or object-level information but without explicitly modeling object
interactions. Thus, they often fail to make visually grounded predictions, and
are sensitive to spurious correlations. In this paper, we propose a novel
spatio-temporal graph model for video captioning that exploits object
interactions in space and time. Our model builds interpretable links and is
able to provide explicit visual grounding. To avoid unstable performance caused
by the variable number of objects, we further propose an object-aware knowledge
distillation mechanism, in which local object information is used to regularize
global scene features. We demonstrate the efficacy of our approach through
extensive experiments on two benchmarks, showing our approach yields
competitive performance with interpretable predictions.

本文提出一种基于时空图模型的视频字幕生成算法，利用物体间相互作用关系提供显式的视觉表征，通过一种对象感知的知识蒸馏机制，可以在不稳定的性能中实现对象的稳定预测。经过对两个基准测试的广泛实验，展示了我们方法的有效性和可解释性预测的竞争性能。