Instructions-tuned Large Language Models (LLMs) gained recently huge
popularity thanks to their ability to interact with users through conversation.
In this work we aim to evaluate their ability to complete multi-turn tasks and
interact with external databases in the context of established task-oriented
dialogue benchmarks. We show that for explicit belief state tracking, LLMs
underperform compared to specialized task-specific models. Nevertheless, they
show ability to guide the dialogue to successful ending if given correct slot
values. Furthermore this ability improves with access to true belief state
distribution or in-domain examples.

本研究旨在研究大型语言模型在多轮任务和与外部数据库交互方面的能力，发现在显式信仰状态跟踪方面，它们表现不如专门的任务特定模型，但是如果给出正确的插槽值，它们表现出将对话引导到成功结局的能力，并且在有真实信仰状态分布或域内示例的情况下，这种能力得到了改进。

LLM 对于面向任务的对话系统是否足够？

Are LLMs All You Need for Task-Oriented Dialogue?

Generative encoder-decoder models offer great promise in developing
domain-general dialog systems. However, they have mainly been applied to
open-domain conversations. This paper presents a practical and novel framework
for building task-oriented dialog systems based on encoder-decoder models. This
framework enables encoder-decoder models to accomplish slot-value independent
decision-making and interact with external databases. Moreover, this paper
shows the flexibility of the proposed method by interleaving chatting
capability with a slot-filling system for better out-of-domain recovery. The
models were trained on both real-user data from a bus information system and
human-human chat data. Results show that the proposed framework achieves good
performance in both offline evaluation metrics and in task success rate with
human users.

本论文提出了一个基于编码器 - 解码器模型的任务导向对话系统的实用且新颖的框架，使其能够完成与外部数据库的交互和独立决策，并将聊天功能与插槽填充系统交替使用，以提高领域外恢复的能力。在实验中，使用公交信息系统的真实用户数据和人际对话数据，所提出的框架表现优异，且离线评估指标和任务成功率与人类用户相当。