Differently from the traditional statistical MT that decomposes the translation task into distinct separately learned components, neural machine translation uses a single neural network to model the entire translation process. Despite neural machine translation being de-facto standard, it is still not clear how NMT models acquire different competences over the course of training, and how this mirrors the different models in traditional SMT. In this work, we look at the competences related to three core SMT components and find that during training, NMT first focuses on learning target-side language modeling, then improves translation quality approaching word-by-word translation, and finally learns more complicated reordering patterns. We show that this behavior holds for several models and language pairs. Additionally, we explain how such an understanding of the training process can be useful in practice and, as an example, show how it can be used to improve vanilla non-autoregressive neural machine translation by guiding teacher model selection.

通过研究 NMT 模型训练过程中的能力表现，发现其在学习目标语言模型、逐词翻译和复杂重排序模式方面的能力表现与传统的 SMT 模型有明显差异，并探讨了这种理解对于优化 NMT 模型的实际应用。

基于经典SMT视角的NMT训练过程：语言建模、词汇翻译、排序