In this paper, we try to understand neural machine translation (NMT) via
simplifying NMT architectures and training encoder-free NMT models. In an
encoder-free model, the sums of word embeddings and positional embeddings
represent the source. The decoder is a standard Transformer or recurrent neural
network that directly attends to embeddings via attention mechanisms.
Experimental results show (1) that the attention mechanism in encoder-free
models acts as a strong feature extractor, (2) that the word embeddings in
encoder-free models are competitive to those in conventional models, (3) that
non-contextualized source representations lead to a big performance drop, and
(4) that encoder-free models have different effects on alignment quality for
German-English and Chinese-English.

本文探讨神经机器翻译 (NMT) 的简化与编码器自由的 NMT 模型。研究结果指出，在编码器自由的模型中，注意力机制作为强特征提取器，并且源表示不可缺少，但词嵌入在编码器自由的模型中是有竞争力的。此外，编码器自由的模型对德英和中英的对齐质量具有不同影响。