Accurately reporting what objects are depicted in an image is largely a
solved problem in automatic caption generation. The next big challenge on the
way to truly humanlike captioning is being able to incorporate the context of
the image and related real world knowledge. We tackle this challenge by
creating an end-to-end caption generation system that makes extensive use of
image-specific encyclopedic data. Our approach includes a novel way of using
image location to identify relevant open-domain facts in an external knowledge
base, with their subsequent integration into the captioning pipeline at both
the encoding and decoding stages. Our system is trained and tested on a new
dataset with naturally produced knowledge-rich captions, and achieves
significant improvements over multiple baselines. We empirically demonstrate
that our approach is effective for generating contextualized captions with
encyclopedic knowledge that is both factually accurate and relevant to the
image.

本文提出了一种使用图像特定的百科全书数据生成上下文化标题的新方法，并通过在编码和解码阶段将其整合到标题生成流程中来使其与图像相关。在新数据集上测试，结果表明该方法能有效地生成准确有关图像的标题。