This paper presents two strong methods, CTC and Imputer, for
non-autoregressive machine translation that model latent alignments with
dynamic programming. We revisit CTC for machine translation and demonstrate
that a simple CTC model can achieve state-of-the-art for single-step
non-autoregressive machine translation, contrary to what prior work indicates.
In addition, we adapt the Imputer model for non-autoregressive machine
translation and demonstrate that Imputer with just 4 generation steps can match
the performance of an autoregressive Transformer baseline. Our latent alignment
models are simpler than many existing non-autoregressive translation baselines;
for example, we do not require target length prediction or re-scoring with an
autoregressive model. On the competitive WMT'14 En$\rightarrow$De task, our CTC
model achieves 25.7 BLEU with a single generation step, while Imputer achieves
27.5 BLEU with 2 generation steps, and 28.0 BLEU with 4 generation steps. This
compares favourably to the autoregressive Transformer baseline at 27.8 BLEU.

本文提出了两种强有力的方法，CTC 和 Imputer，用于非自回归机器翻译，模拟了动态规划中的潜在对齐。我们重新审视了 CTC 和 Imputer 模型，并证明了一个简单的 CTC 模型可以实现单步非自回归机器翻译的最新成果，与之前的工作相反。另外，我们还将 Imputer 模型适应于非自回归机器翻译，并证明只需 4 个生成步骤，Imputer 即可匹配自回归 Transformer 基准的性能。

具有潜在对齐的非自回归机器翻译

Non-Autoregressive Machine Translation with Latent Alignments

This paper presents the Imputer, a neural sequence model that generates
output sequences iteratively via imputations. The Imputer is an iterative
generative model, requiring only a constant number of generation steps
independent of the number of input or output tokens. The Imputer can be trained
to approximately marginalize over all possible alignments between the input and
output sequences, and all possible generation orders. We present a tractable
dynamic programming training algorithm, which yields a lower bound on the log
marginal likelihood. When applied to end-to-end speech recognition, the Imputer
outperforms prior non-autoregressive models and achieves competitive results to
autoregressive models. On LibriSpeech test-other, the Imputer achieves 11.1
WER, outperforming CTC at 13.0 WER and seq2seq at 12.5 WER.

本文提出 Imputer，一种神经序列模型，通过插值迭代地生成输出序列。Imputer 是一种迭代生成模型，只需要恒定数量的生成步骤，与输入或输出标记的数量无关。Imputer 可以被训练成近似边际化输入和输出序列之间所有可能的对齐和所有可能的生成顺序等。我们提出了一种可行的动态规划训练算法，它产生对数边际似然的下界。在端到端语音识别中，Imputer 优于之前的非自回归模型，并与自回归模型取得竞争性的结果。在 LibriSpeech test-other 上，Imputer 取得了 11.1 WER 的成绩，优于 13.0 WER 的 CTC 和 12.5 WER 的 seq2seq。