Undirected neural sequence models have achieved performance competitive with
the state-of-the-art directed sequence models that generate monotonically from
left to right in machine translation tasks. In this work, we train a policy
that learns the generation order for a pre-trained, undirected translation
model via reinforcement learning. We show that the translations decoded by our
learned orders achieve higher BLEU scores than the outputs decoded from left to
right or decoded by the learned order from Mansimov et al. (2019) on the WMT'14
German-English translation task. On examples with a maximum source and target
length of 30 from De-En, WMT'16 English-Romanian, and WMT'21 English-Chinese
translation tasks, our learned order outperforms all heuristic generation
orders on four out of six tasks. We next carefully analyze the learned order
patterns via qualitative and quantitative analysis. We show that our policy
generally follows an outer-to-inner order, predicting the left-most and
right-most positions first, and then moving toward the middle while skipping
less important words at the beginning. Furthermore, the policy usually predicts
positions for a single syntactic constituent structure in consecutive steps. We
believe our findings could provide more insights on the mechanism of undirected
generation models and encourage further research in this direction. Our code is
publicly available at this https URL

通过强化学习训练预训练的无向翻译模型的生成顺序，从而实现了比从左到右或学习自 Mansimov 等人 (2019) 学习的顺序得到的输出 BLEU 分数更高的翻译结果。

无向序列模型生成顺序的学习和分析

Learning and Analyzing Generation Order for Undirected Sequence Models

Undirected neural sequence models such as BERT (Devlin et al., 2019) have
received renewed interest due to their success on discriminative natural
language understanding tasks such as question-answering and natural language
inference. The problem of generating sequences directly from these models has
received relatively little attention, in part because generating from
undirected models departs significantly from conventional monotonic generation
in directed sequence models. We investigate this problem by proposing a
generalized model of sequence generation that unifies decoding in directed and
undirected models. The proposed framework models the process of generation
rather than the resulting sequence, and under this framework, we derive various
neural sequence models as special cases, such as autoregressive,
semi-autoregressive, and refinement-based non-autoregressive models. This
unification enables us to adapt decoding algorithms originally developed for
directed sequence models to undirected sequence models. We demonstrate this by
evaluating various handcrafted and learned decoding strategies on a BERT-like
machine translation model (Lample & Conneau, 2019). The proposed approach
achieves constant-time translation results on par with linear-time translation
results from the same undirected sequence model, while both are competitive
with the state-of-the-art on WMT'14 English-German translation.

提出使用序列生成的方法来解决从无向神经序列模型（如 BERT）中生成序列的问题，并将其与定向的序列模型进行统一，从而能够使用原来专门为定向序列模型开发的解码算法在无向序列模型中进行机器翻译。在 WMT'14 英德翻译任务上，提出的方法取得了类似于线性时间翻译结果的恒定时间结果，并且与最先进的方法相媲美。