Multimodal machine translation (MMT) aims to improve translation quality by
incorporating information from other modalities, such as vision. Previous MMT
systems mainly focus on better access and use of visual information and tend to
validate their methods on image-related datasets. These studies face two
challenges. First, they can only utilize triple data (bilingual texts with
images), which is scarce; second, current benchmarks are relatively restricted
and do not correspond to realistic scenarios. Therefore, this paper
correspondingly establishes new methods and new datasets for MMT. First, we
propose a framework 2/3-Triplet with two new approaches to enhance MMT by
utilizing large-scale non-triple data: monolingual image-text data and parallel
text-only data. Second, we construct an English-Chinese {e}-commercial
{m}ulti{m}odal {t}ranslation dataset (including training and testing), named
EMMT, where its test set is carefully selected as some words are ambiguous and
shall be translated mistakenly without the help of images. Experiments show
that our method is more suitable for real-world scenarios and can significantly
improve translation performance by using more non-triple data. In addition, our
model also rivals various SOTA models in conventional multimodal translation
benchmarks.

通过提出一个新的方法和新的数据集，本文致力于提高多模式机器翻译的翻译质量并解决了只能利用少量数据和当前的基准相对较为受限约束的问题，结果表明我们的方法更适合实际场景并显著提高了翻译性能。