Most of the Neural Machine Translation (NMT) models are based on the sequence-to-sequence (Seq2Seq) model with an encoder-decoder framework equipped with the attention mechanism. However, the conventional attention mechanism treats the decoding at each time step equally with the same matrix, which is problematic since the softness of the attention for different types of words (e.g. content words and function words) should differ. Therefore, we propose a new model with a mechanism called Self-Adaptive Control of Temperature (SACT) to control the softness of attention by means of an attention temperature. Experimental results on the Chinese-English translation and English-Vietnamese translation demonstrate that our model outperforms the baseline models, and the analysis and the case study show that our model can attend to the most relevant elements in the source-side contexts and generate the translation of high quality.

本文介绍了一种使用自适应控温机制的Seq2Seq模型，以此来改进attention机制，在汉英和英越翻译任务中取得了优异的表现，研究表明这种改进能够提高翻译的质量且能够关注源文本中最相关的元素。

自适应注意力温度的深度神经机器翻译集中与分散注意力控制学习