Recent progress on automatic generation of image captions has shown that it
is possible to describe the most salient information conveyed by images with
accurate and meaningful sentences. In this paper, we propose an image caption
system that exploits the parallel structures between images and sentences. In
our model, the process of generating the next word, given the previously
generated ones, is aligned with the visual perception experience where the
attention shifting among the visual regions imposes a thread of visual
ordering. This alignment characterizes the flow of "abstract meaning", encoding
what is semantically shared by both the visual scene and the text description.
Our system also makes another novel modeling contribution by introducing
scene-specific contexts that capture higher-level semantic information encoded
in an image. The contexts adapt language models for word generation to specific
scene types. We benchmark our system and contrast to published results on
several popular datasets. We show that using either region-based attention or
scene-specific contexts improves systems without those components. Furthermore,
combining these two modeling ingredients attains the state-of-the-art
performance.

本文提出了一种使用平行结构、利用图像和句子之间的相似性及上下文信息生成图像描述的方法，将视觉感知与文本描述相结合，使用场景特定的上下文信息对句子中的词语生成进行调整，在多个流行数据集上的实验表明本方法在生成精准、丰富的图像描述方面表现优异。