Though early successes of Statistical Machine Translation (SMT) systems are
attributed in part to the explicit modelling of the interaction between any two
source and target units, e.g., alignment, the recent Neural Machine Translation
(NMT) systems resort to the attention which partially encodes the interaction
for efficiency. In this paper, we employ Joint Representation that fully
accounts for each possible interaction. We sidestep the inefficiency issue by
refining representations with the proposed efficient attention operation. The
resulting Reformer models offer a new Sequence-to- Sequence modelling paradigm
besides the Encoder-Decoder framework and outperform the Transformer baseline
in either the small scale IWSLT14 German-English, English-German and IWSLT15
Vietnamese-English or the large scale NIST12 Chinese-English translation tasks
by about 1 BLEU point.We also propose a systematic model scaling approach,
allowing the Reformer model to beat the state-of-the-art Transformer in IWSLT14
German-English and NIST12 Chinese-English with about 50% fewer parameters. The
code is publicly available at this https URL

本研究提出一种基于联合表示的神经机器翻译模型，通过提出的高效注意力机制对表示进行精细化处理，实现了新的序列到序列建模范式并在多项机器翻译任务中取得了更优结果，同时提出了系统的模型放大方法，成功将模型规模缩小 50%，同时取得更高的翻译品质。