Unsupervised machine translation (MT) has recently achieved impressive
results with monolingual corpora only. However, it is still challenging to
associate source-target sentences in the latent space. As people speak
different languages biologically share similar visual systems, the potential of
achieving better alignment through visual content is promising yet
under-explored in unsupervised multimodal MT (MMT). In this paper, we
investigate how to utilize visual content for disambiguation and promoting
latent space alignment in unsupervised MMT. Our model employs multimodal
back-translation and features pseudo visual pivoting in which we learn a shared
multilingual visual-semantic embedding space and incorporate visually-pivoted
captioning as additional weak supervision. The experimental results on the
widely used Multi30K dataset show that the proposed model significantly
improves over the state-of-the-art methods and generalizes well when the images
are not available at the testing time.

本研究探讨如何利用视觉内容实现无监督多模态机器翻译领域的降歧和提升潜空间的对齐度。该模型采用多模态反向翻译，具备伪视觉枢轴功能，实现了多语言视觉 - 语义嵌入空间学习和视觉轴描述补充弱监督。实验证明该模型显著超越了最先进的方法，并能在测试时很好地进行泛化。