Non-autoregressive Transformer is a promising text generation model. However,
current non-autoregressive models still fall behind their autoregressive
counterparts in translation quality. We attribute this accuracy gap to the lack
of dependency modeling among decoder inputs. In this paper, we propose CNAT,
which learns implicitly categorical codes as latent variables into the
non-autoregressive decoding. The interaction among these categorical codes
remedies the missing dependencies and improves the model capacity. Experiment
results show that our model achieves comparable or better performance in
machine translation tasks, compared with several strong baselines.

本文提出了一种将 CNAT 学习的分类代码作为潜变量引入非自回归解码器的方法，从而解决其输入依赖性建模缺乏的限制，实现机器翻译任务中与多个强基线相比相当或更好的性能。

通过学习目标分类编码实现非自回归式翻译

Non-Autoregressive Translation by Learning Target Categorical Codes

Non-autoregressive translation (NAT) models, which remove the dependence on
previous target tokens from the inputs of the decoder, achieve significantly
inference speedup but at the cost of inferior accuracy compared to
autoregressive translation (AT) models. Previous work shows that the quality of
the inputs of the decoder is important and largely impacts the model accuracy.
In this paper, we propose two methods to enhance the decoder inputs so as to
improve NAT models. The first one directly leverages a phrase table generated
by conventional SMT approaches to translate source tokens to target tokens,
which are then fed into the decoder as inputs. The second one transforms
source-side word embeddings to target-side word embeddings through
sentence-level alignment and word-level adversary learning, and then feeds the
transformed word embeddings into the decoder as inputs. Experimental results
show our method largely outperforms the NAT baseline~\citep{gu2017non} by
$5.11$ BLEU scores on WMT14 English-German task and $4.72$ BLEU scores on WMT16
English-Romanian task.

本文提出了两种增强解码器输入以提高 NAT 模型的翻译准确性的方法，并表明这些方法在 WMT14 英德任务和 WMT16 英罗马任务中比 NAT 基线高出 $5.11$ BLEU 分数和 $4.72$ BLEU 分数。