This paper presents a novel approach for automatically generating image
descriptions: visual detectors, language models, and multimodal similarity
models learnt directly from a dataset of image captions. We use multiple
instance learning to train visual detectors for words that commonly occur in
captions, including many different parts of speech such as nouns, verbs, and
adjectives. The word detector outputs serve as conditional inputs to a
maximum-entropy language model. The language model learns from a set of over
400,000 image descriptions to capture the statistics of word usage. We capture
global semantics by re-ranking caption candidates using sentence-level features
and a deep multimodal similarity model. Our system is state-of-the-art on the
official Microsoft COCO benchmark, producing a BLEU-4 score of 29.1%. When
human judges compare the system captions to ones written by other people on our
held-out test set, the system captions have equal or better quality 34% of the
time.

本文提出了一种用于自动生成图像描述的新方法：使用从图像标题数据集中直接学习的视觉探测器、语言模型和多模式相似模型。