Reinforcement learning (RL) is a powerful approach to enhance task-oriented
dialogue (TOD) systems. However, existing RL methods tend to mainly focus on
generation tasks, such as dialogue policy learning (DPL) or response generation
(RG), while neglecting dialogue state tracking (DST) for understanding. This
narrow focus limits the systems to achieve globally optimal performance by
overlooking the interdependence between understanding and generation.
Additionally, RL methods face challenges with sparse and delayed rewards, which
complicates training and optimization. To address these issues, we extend RL
into both understanding and generation tasks by introducing step-by-step
rewards throughout the token generation. The understanding reward increases as
more slots are correctly filled in DST, while the generation reward grows with
the accurate inclusion of user requests. Our approach provides a balanced
optimization aligned with task completion. Experimental results demonstrate
that our approach effectively enhances the performance of TOD systems and
achieves new state-of-the-art results on three widely used datasets, including
MultiWOZ2.0, MultiWOZ2.1, and In-Car. Our approach also shows superior few-shot
ability in low-resource settings compared to current models.

通过引入逐步奖励机制，我们的方法在理解和生成任务中对强化学习进行了扩展，实现了平衡优化，提高了任务导向对话系统的性能，并在包括 MultiWOZ2.0、MultiWOZ2.1 和 In-Car 在内的三个广泛使用的数据集上取得了新的最先进结果。与现有模型相比，我们的方法还展现了在低资源环境中的出色少样本能力。