There are common semantics shared across text and images. Given a sentence in
a source language, whether depicting the visual scene helps translation into a
target language? Existing multimodal neural machine translation methods (MNMT)
require triplets of bilingual sentence - image for training and tuples of
source sentence - image for inference. In this paper, we propose ImagiT, a
novel machine translation method via visual imagination. ImagiT first learns to
generate visual representation from the source sentence, and then utilizes both
source sentence and the "imagined representation" to produce a target
translation. Unlike previous methods, it only needs the source sentence at the
inference time. Experiments demonstrate that ImagiT benefits from visual
imagination and significantly outperforms the text-only neural machine
translation baselines. Further analysis reveals that the imagination process in
ImagiT helps fill in missing information when performing the degradation
strategy.

本文提出一种基于视觉想象的神经机器翻译方法 ImagiT，仅需要源语言句子即可生成目标翻译，并证明该方法较传统文本翻译方法表现更好，揭示想象过程在修复缺失信息时的重要性。