Large Language Models (LLMs), such as ChatGPT, are becoming increasingly
sophisticated, demonstrating capabilities that closely resemble those of
humans. These AI models are playing an essential role in assisting humans with
a wide array of tasks in daily life. A significant application of AI is its use
as a chat agent, responding to human inquiries across various domains. Current
LLMs have shown proficiency in answering general questions. However, basic
question-answering dialogue often falls short in complex diagnostic scenarios,
such as legal or medical consultations. These scenarios typically necessitate
Task-Oriented Dialogue (TOD), wherein an AI chat agent needs to proactively
pose questions and guide users towards specific task completion. Previous
fine-tuning models have underperformed in TOD, and current LLMs do not
inherently possess this capability. In this paper, we introduce DiagGPT
(Dialogue in Diagnosis GPT), an innovative method that extends LLMs to TOD
scenarios. Our experiments reveal that DiagGPT exhibits outstanding performance
in conducting TOD with users, demonstrating its potential for practical
applications.

本文介绍了一种创新方法 DiagGPT，将大型语言模型（LLMs）扩展到任务导向对话（TOD）场景，并通过实验证明 DiagGPT 在 TOD 方面表现出色，显示出实际应用的潜力。

DiagGPT：面向任务导向对话的基于 LLM 的聊天机器人及自动话题管理

DiagGPT: An LLM-based Chatbot with Automatic Topic Management for  Task-Oriented Dialogue

While deep learning succeeds in a wide range of tasks, it highly depends on
the massive collection of annotated data which is expensive and time-consuming.
To lower the cost of data annotation, active learning has been proposed to
interactively query an oracle to annotate a small proportion of informative
samples in an unlabeled dataset. Inspired by the fact that the samples with
higher loss are usually more informative to the model than the samples with
lower loss, in this paper we present a novel deep active learning approach that
queries the oracle for data annotation when the unlabeled sample is believed to
incorporate high loss. The core of our approach is a measurement Temporal
Output Discrepancy (TOD) that estimates the sample loss by evaluating the
discrepancy of outputs given by models at different optimization steps. Our
theoretical investigation shows that TOD lower-bounds the accumulated sample
loss thus it can be used to select informative unlabeled samples. On basis of
TOD, we further develop an effective unlabeled data sampling strategy as well
as an unsupervised learning criterion for active learning. Due to the
simplicity of TOD, our methods are efficient, flexible, and task-agnostic.
Extensive experimental results demonstrate that our approach achieves superior
performances than the state-of-the-art active learning methods on image
classification and semantic segmentation tasks. In addition, we show that TOD
can be utilized to select the best model of potentially the highest testing
accuracy from a pool of candidate models.

该论文提出了一种基于 TOD 测量方法的深度主动学习方法，通过查询 oracle 注释含有高损失的未标记样本以降低数据注释成本，并在图像分类和语义分割任务上展示出卓越的性能。

基于损失估计的主动学习中的时间输出差异

Temporal Output Discrepancy for Loss Estimation-based Active Learning

Pre-trained language models (PLM) have advanced the state-of-the-art across
NLP applications, but lack domain-specific knowledge that does not naturally
occur in pre-training data. Previous studies augmented PLMs with symbolic
knowledge for different downstream NLP tasks. However, knowledge bases (KBs)
utilized in these studies are usually large-scale and static, in contrast to
small, domain-specific, and modifiable knowledge bases that are prominent in
real-world task-oriented dialogue (TOD) systems. In this paper, we showcase the
advantages of injecting domain-specific knowledge prior to fine-tuning on TOD
tasks. To this end, we utilize light-weight adapters that can be easily
integrated with PLMs and serve as a repository for facts learned from different
KBs. To measure the efficacy of proposed knowledge injection methods, we
introduce Knowledge Probing using Response Selection (KPRS) -- a probe designed
specifically for TOD models. Experiments on KPRS and the response generation
task show improvements of knowledge injection with adapters over strong
baselines.

本研究中，我们使用轻量级 adapters 来注入特定领域的知识以在 TOD 任务上进行微调，通过 KPRS 探测方法证明 adapters 对于对话生成任务有着优秀的表现。

将领域知识注入语言模型以用于面向任务的对话系统

Injecting Domain Knowledge in Language Models for Task-Oriented Dialogue  Systems

We present BotSIM, a data-efficient end-to-end Bot SIMulation toolkit for
commercial text-based task-oriented dialog (TOD) systems. BotSIM consists of
three major components: 1) a Generator that can infer semantic-level dialog
acts and entities from bot definitions and generate user queries via
model-based paraphrasing; 2) an agenda-based dialog user Simulator (ABUS) to
simulate conversations with the dialog agents; 3) a Remediator to analyze the
simulated conversations, visualize the bot health reports and provide
actionable remediation suggestions for bot troubleshooting and improvement. We
demonstrate BotSIM's effectiveness in end-to-end evaluation, remediation and
multi-intent dialog generation via case studies on two commercial bot
platforms. BotSIM's "generation-simulation-remediation" paradigm accelerates
the end-to-end bot evaluation and iteration process by: 1) reducing manual test
cases creation efforts; 2) enabling a holistic gauge of the bot in terms of NLU
and end-to-end performance via extensive dialog simulation; 3) improving the
bot troubleshooting process with actionable suggestions. A demo of our system
can be found at this https URL and a demo video at
this https URL. We have open-sourced the toolkit at
this https URL

BotSIM 是一种数据高效的端到端 Bot 模拟工具包，用于商业文本任务导向对话系统，其包括生成器、基于议程的对话用户模拟器和修复程序，加速端到端 Bot 评估和迭代过程。