Image captioning has emerged as an interesting research field in recent years
due to its broad application scenarios. The traditional paradigm of image
captioning relies on paired image-caption datasets to train the model in a
supervised manner. However, creating such paired datasets for every target
language is prohibitively expensive, which hinders the extensibility of
captioning technology and deprives a large part of the world population of its
benefit. In this work, we present a novel unpaired cross-lingual method to
generate image captions without relying on any caption corpus in the source or
the target language. Specifically, our method consists of two phases: (i) a
cross-lingual auto-encoding process, which utilizing a sentence parallel
(bitext) corpus to learn the mapping from the source to the target language in
the scene graph encoding space and decode sentences in the target language, and
(ii) a cross-modal unsupervised feature mapping, which seeks to map the encoded
scene graph features from image modality to language modality. We verify the
effectiveness of our proposed method on the Chinese image caption generation
task. The comparisons against several existing methods demonstrate the
effectiveness of our approach.

本文提出了一种新颖的基于跨语言无配对数据的图像描述生成方法，通过跨语言的自动编码和跨媒体非监督特征映射实现从图像模态到语言模态的生成， 并在汉语图像描述生成任务上展示了其有效性。