During task-oriented dialogues (TODs), human users naturally introduce
chitchat that is beyond the immediate scope of the task, interfering with the
flow of the conversation. To address this issue without the need for expensive
manual data creation, we use few-shot prompting with Llama-2-70B to enhance the
MultiWOZ dataset with user backstories, a typical example of chitchat
interference in TODs. We assess the impact of this addition by testing two
models: one trained solely on TODs and another trained on TODs with a
preliminary chitchat interaction. Our analysis reveals that our enriched
dataset poses a significant challenge to these systems. Moreover, we
demonstrate that our dataset can be effectively used for training purposes,
enabling a system to consistently acknowledge the user's backstory while also
successfully moving the task forward in the same turn, as confirmed by human
evaluation. These findings highlight the benefits of generating novel
chitchat-TOD scenarios to test TOD systems more thoroughly and improve their
resilience to natural user interferences.

通过使用 Llama-2-70B 进行少样本提示来提高 MultiWOZ 数据集中用户故事的质量，我们评估了这种添加对模型的影响，结果显示这种丰富的数据集给系统带来了显著挑战，同时证明了该数据集可用于训练，能够在同一轮中始终自觉承认用户的背景故事并成功推进任务，这些发现强调了生成新的聊天 - 任务导向对话场景以更全面地测试 TOD 系统并提高其对自然用户干扰的适应能力的好处。