The past year has witnessed rapid advances in sequence-to-sequence (seq2seq)
modeling for Machine Translation (MT). The classic RNN-based approaches to MT
were first out-performed by the convolutional seq2seq model, which was then
out-performed by the more recent Transformer model. Each of these new
approaches consists of a fundamental architecture accompanied by a set of
modeling and training techniques that are in principle applicable to other
seq2seq architectures. In this paper, we tease apart the new architectures and
their accompanying techniques in two ways. First, we identify several key
modeling and training techniques, and apply them to the RNN architecture,
yielding a new RNMT+ model that outperforms all of the three fundamental
architectures on the benchmark WMT'14 English to French and English to German
tasks. Second, we analyze the properties of each fundamental seq2seq
architecture and devise new hybrid architectures intended to combine their
strengths. Our hybrid models obtain further improvements, outperforming the
RNMT+ model on both benchmark datasets.

本文研究序列到序列 (seq2seq) 建模在机器翻译 (MT) 中的最新方法，提出了一种新的 RNN 模型 (RNMT+), 并且通过分析每种常见 seq2seq 架构的特点，设计出了一些混合架构来综合它们的优点，实验结果表明，混合架构进一步提高了 MT 的翻译性能。