Grounded video description (GVD) encourages captioning models to attend to
appropriate video regions (e.g., objects) dynamically and generate a
description. Such a setting can help explain the decisions of captioning models
and prevents the model from hallucinating object words in its description.
However, such design mainly focuses on object word generation and thus may
ignore fine-grained information and suffer from missing visual concepts.
Moreover, relational words (e.g., "jump left or right") are usual
spatio-temporal inference results, i.e., these words cannot be grounded on
certain spatial regions. To tackle the above limitations, we design a novel
relational graph learning framework for GVD, in which a language-refined scene
graph representation is designed to explore fine-grained visual concepts.
Furthermore, the refined graph can be regarded as relational inductive
knowledge to assist captioning models in selecting the relevant information it
needs to generate correct words. We validate the effectiveness of our model
through automatic metrics and human evaluation, and the results indicate that
our approach can generate more fine-grained and accurate description, and it
solves the problem of object hallucination to some extent.

本研究设计了一种新颖的关系图学习框架，包括一种语言细化的场景图表示，探索细粒度的视觉概念，并解决了对象幻觉问题。实验结果表明，本方法可以生成更精细和准确的描述。