This paper explores SynTOD, a new synthetic data generation approach for
developing end-to-end Task-Oriented Dialogue (TOD) Systems capable of handling
complex tasks such as intent classification, slot filling, conversational
question-answering, and retrieval-augmented response generation, without
relying on crowdsourcing or real-world data. SynTOD utilizes a state transition
graph to define the desired behavior of a TOD system and generates diverse,
structured conversations through random walks and response simulation using
large language models (LLMs). In our experiments, using graph-guided response
simulations leads to significant improvements in intent classification, slot
filling and response relevance compared to naive single-prompt simulated
conversations. We also investigate the end-to-end TOD effectiveness of
different base and instruction-tuned LLMs, with and without the constructed
synthetic conversations. Finally, we explore how various LLMs can evaluate
responses in a TOD system and how well they are correlated with human
judgments. Our findings pave the path towards quick development and evaluation
of domain-specific TOD systems. We release our datasets, models, and code for
research purposes.

该研究探讨了 SynTOD，一种用于开发端到端任务导向对话系统的新型合成数据生成方法，该方法能够处理意图分类、槽填充、对话问答和检索增益响应生成等复杂任务，而无需依赖众包或现实世界的数据。实验结果显示，使用受图引导的响应模拟能够显著提高意图分类、槽填充和响应相关性，相较于单一提示的模拟对话。该研究还调查了不同基础和指导调整的大型语言模型在端到端任务导向对话系统上的有效性，并探索了它们在评估回应和与人类判断之间的相关性。这些发现为领域特定的任务导向对话系统的快速开发和评估铺平了道路。研究团队还发布了用于研究目的的数据集、模型和代码。