Scene graph generation has received growing attention with the advancements in image understanding tasks such as object detection, attributes and relationship prediction,~\etc. However, existing datasets are biased in terms of object and relationship labels, or often come with noisy and missing annotations, which makes the development of a reliable scene graph prediction model very challenging. In this paper, we propose a novel scene graph generation algorithm with external knowledge and image reconstruction loss to overcome these dataset issues. In particular, we extract commonsense knowledge from the external knowledge base to refine object and phrase features for improving generalizability in scene graph generation. To address the bias of noisy object annotations, we introduce an auxiliary image reconstruction path to regularize the scene graph generation network. Extensive experiments show that our framework can generate better scene graphs, achieving the state-of-the-art performance on two benchmark datasets: Visual Relationship Detection and Visual Genome datasets.

本文提出了一种利用外部知识和图像重建损失的新型场景图生成算法，提取外部常识知识以改进场景图的可推广性，并通过在场景图生成网络上引入辅助图像重建路径来解决嘈杂对象注释的偏差问题，全面实验证明我们的框架可以生成更好的场景图，并在“视觉关系检测”和“视觉基因组”两个基准数据集上实现了最先进的性能。

利用外部知识和图像重建的场景图生成