Unsupervised image captioning with no annotations is an emerging challenge in computer vision, where the existing arts usually adopt GAN (Generative Adversarial Networks) models. In this paper, we propose a novel memory-based network rather than GAN, named Recurrent Relational Memory Network ($R^2M$). Unlike complicated and sensitive adversarial learning that non-ideally performs for long sentence generation, $R^2M$ implements a concepts-to-sentence memory translator through two-stage memory mechanisms: fusion and recurrent memories, correlating the relational reasoning between common visual concepts and the generated words for long periods. $R^2M$ encodes visual context through unsupervised training on images, while enabling the memory to learn from irrelevant textual corpus via supervised fashion. Our solution enjoys less learnable parameters and higher computational efficiency than GAN-based methods, which heavily bear parameter sensitivity. We experimentally validate the superiority of $R^2M$ than state-of-the-arts on all benchmark datasets.

本文提出了一种基于记忆机制的无监督图像描述模型——循环关系记忆网络(R2M)，相较于GAN模型，该方法克服了句子生成中复杂和敏感的对抗学习，并且通过融合和循环两种记忆机制实现了从通用可视化概念到自然语言的翻译；我们在多项基准数据集上的实验证明，R2M在参数少、计算效率高等各方面均优于目前最先进的技术。

基于循环关系记忆网络的无监督图像描述生成