Neural machine translation (NMT) generates the next target token given as
input the previous ground truth target tokens during training while the
previous generated target tokens during inference, which causes discrepancy
between training and inference as well as error propagation, and affects the
translation accuracy. In this paper, we introduce an error correction mechanism
into NMT, which corrects the error information in the previous generated tokens
to better predict the next token. Specifically, we introduce two-stream
self-attention from XLNet into NMT decoder, where the query stream is used to
predict the next token, and meanwhile the content stream is used to correct the
error information from the previous predicted tokens. We leverage scheduled
sampling to simulate the prediction errors during training. Experiments on
three IWSLT translation datasets and two WMT translation datasets demonstrate
that our method achieves improvements over Transformer baseline and scheduled
sampling. Further experimental analyses also verify the effectiveness of our
proposed error correction mechanism to improve the translation quality.

本文提出了一种将错误校正机制引入神经机器翻译中的方法，通过使用两个流的自注意力机制，在保证预测下一个标记的同时，对前一个标记的错误信息进行更正，使用预测偏差模拟训练，并在多个数据集上进行了实验验证，证明了其在提高翻译质量方面的有效性。