Non-autoregressive Transformers (NATs) significantly reduce the decoding
latency by generating all tokens in parallel. However, such independent
predictions prevent NATs from capturing the dependencies between the tokens for
generating multiple possible translations. In this paper, we propose Directed
Acyclic Transfomer (DA-Transformer), which represents the hidden states in a
Directed Acyclic Graph (DAG), where each path of the DAG corresponds to a
specific translation. The whole DAG simultaneously captures multiple
translations and facilitates fast predictions in a non-autoregressive fashion.
Experiments on the raw training data of WMT benchmark show that DA-Transformer
substantially outperforms previous NATs by about 3 BLEU on average, which is
the first NAT model that achieves competitive results with autoregressive
Transformers without relying on knowledge distillation.

通过将隐藏状态表示成有向无环图，在非自回归的情况下同时捕获多个翻译，并促进快速预测，DA-Transformer 在 WMT 基准测试的原始训练数据上实验表明，DA-Transformer 比以前的 NATs（非自回归 transform）平均优于 3 BLEU，是第一个在不依赖于知识蒸馏的情况下实现与自回归 Transformer 竞争性结果的 NAT 模型。