Multimodal machine translation (MMT) aims to improve translation quality by
equipping the source sentence with its corresponding image. Despite the
promising performance, MMT models still suffer the problem of input
degradation: models focus more on textual information while visual information
is generally overlooked. In this paper, we endeavor to improve MMT performance
by increasing visual awareness from an information theoretic perspective. In
detail, we decompose the informative visual signals into two parts:
source-specific information and target-specific information. We use mutual
information to quantify them and propose two methods for objective optimization
to better leverage visual signals. Experiments on two datasets demonstrate that
our approach can effectively enhance the visual awareness of MMT model and
achieve superior results against strong baselines.

本文旨在从信息论角度提高多模式机器翻译的视觉感知能力，通过将信息量化为源特定信息和目标特定信息，并提出两种目标优化方法以更好地利用视觉信号来解决输入退化的问题。实验结果表明，我们的方法可以显著提高 MMT 模型的视觉感知能力，并在两个数据集上取得了优秀的结果。