Multilingual translation supports multiple translation directions by
projecting all languages in a shared space, but the translation quality is
undermined by the difference between languages in the text-only modality,
especially when the number of languages is large. To bridge this gap, we
introduce visual context as the universal language-independent representation
to facilitate multilingual translation. In this paper, we propose a framework
to leverage the multimodal prompt to guide the Multimodal Multilingual neural
Machine Translation (m3P), which aligns the representations of different
languages with the same meaning and generates the conditional vision-language
memory for translation. We construct a multilingual multimodal instruction
dataset (InstrMulti102) to support 102 languages. Our method aims to minimize
the representation distance of different languages by regarding the image as a
central language. Experimental results show that m3P outperforms previous
text-only baselines and multilingual multimodal methods by a large margin.
Furthermore, the probing experiments validate the effectiveness of our method
in enhancing translation under the low-resource and massively multilingual
scenario.

通过引入视觉语境作为通用的语言无关表示来促进多语言翻译的方法，该方法利用多模态提示来指导多模态多语言神经机器翻译，在低资源和大规模多语言场景下改善了翻译质量。