Grounding language to visual relations is critical to various
language-and-vision applications. In this work, we tackle two fundamental
language-and-vision tasks: image-text matching and image captioning, and
demonstrate that neural scene graph generators can learn effective visual
relation features to facilitate grounding language to visual relations and
subsequently improve the two end applications. By combining relation features
with the state-of-the-art models, our experiments show significant improvement
on the standard Flickr30K and MSCOCO benchmarks. Our experimental results and
analysis show that relation features improve downstream models' capability of
capturing visual relations in end vision-and-language applications. We also
demonstrate the importance of learning scene graph generators with visually
relevant relations to the effectiveness of relation features.

通过结合神经场景图生成器和最先进的模型，我们的实验表明用于促进语言到视觉关系的关系特征能够显著改进标准的 Flickr30K 和 MSCOCO 基准测试，在端到端的视觉和语言应用中捕获视觉关系。