Non-autoregressive (NAR) language models are known for their low latency in
neural machine translation (NMT). However, a performance gap exists between NAR
and autoregressive models due to the large decoding space and difficulty in
capturing dependency between target words accurately. Compounding this,
preparing appropriate training data for NAR models is a non-trivial task, often
exacerbating exposure bias. To address these challenges, we apply reinforcement
learning (RL) to Levenshtein Transformer, a representative edit-based NAR
model, demonstrating that RL with self-generated data can enhance the
performance of edit-based NAR models. We explore two RL approaches: stepwise
reward maximization and episodic reward maximization. We discuss the respective
pros and cons of these two approaches and empirically verify them. Moreover, we
experimentally investigate the impact of temperature setting on performance,
confirming the importance of proper temperature setting for NAR models'
training.

非自回归（NAR）语言模型在神经机器翻译（NMT）中以低延迟著称。然而，由于较大的解码空间和准确捕捉目标单词间的依赖关系的困难，在 NAR 模型和自回归模型之间存在性能差距。为了解决这些挑战，我们将强化学习（RL）应用于编辑型 NAR 模型的典型例子 Levenshtein Transformer，通过使用自生成的数据，证明 RL 可以提高编辑型 NAR 模型的性能。我们探索了两种 RL 方法：逐步奖励最大化和情节奖励最大化，讨论了这两种方法的优缺点并进行了实证验证。此外，我们在实验中研究了温度设置对性能的影响，确认正确的温度设置对 NAR 模型的训练的重要性。