Recently, image captioning has aroused great interest in both academic and
industrial worlds. Most existing systems are built upon large-scale datasets
consisting of image-sentence pairs, which, however, are time-consuming to
construct. In addition, even for the most advanced image captioning systems, it
is still difficult to realize deep image understanding. In this work, we
achieve unpaired image captioning by bridging the vision and the language
domains with high-level semantic information. The motivation stems from the
fact that the semantic concepts with the same modality can be extracted from
both images and descriptions. To further improve the quality of captions
generated by the model, we propose the Semantic Relationship Explorer, which
explores the relationships between semantic concepts for better understanding
of the image. Extensive experiments on MSCOCO dataset show that we can generate
desirable captions without paired datasets. Furthermore, the proposed approach
boosts five strong baselines under the paired setting, where the most
significant improvement in CIDEr score reaches 8%, demonstrating that it is
effective and generalizes well to a wide range of models.

本文提出了一种跨越视觉和语言领域的高级语义信息桥接技术，以实现无配对图像说明，并引入了基于语义概念关系探索的解决方案 ——Semantic Relationship Explorer，能够有效地提高图像说明的质量，并且在 MSCOCO 数据集下总体得分比配对数据集高出 8%。