Out-of-domain (OOD) intent detection aims to examine whether the user's query
falls outside the predefined domain of the system, which is crucial for the
proper functioning of task-oriented dialogue (TOD) systems. Previous methods
address it by fine-tuning discriminative models. Recently, some studies have
been exploring the application of large language models (LLMs) represented by
ChatGPT to various downstream tasks, but it is still unclear for their ability
on OOD detection task.This paper conducts a comprehensive evaluation of LLMs
under various experimental settings, and then outline the strengths and
weaknesses of LLMs. We find that LLMs exhibit strong zero-shot and few-shot
capabilities, but is still at a disadvantage compared to models fine-tuned with
full resource. More deeply, through a series of additional analysis
experiments, we discuss and summarize the challenges faced by LLMs and provide
guidance for future work including injecting domain knowledge, strengthening
knowledge transfer from IND(In-domain) to OOD, and understanding long
instructions.

对大规模语言模型在超出预定义领域内意图检测任务中的能力进行了综合评估，揭示了它们的优势和劣势，发现 LLMs 表现出强大的零样本和少样本能力，但与使用完整资源进行微调的模型相比仍存在劣势。通过进一步的分析实验，讨论总结了 LLMs 面临的挑战，并提出了对未来工作的指导，包括注入领域知识、加强从领域内到超领域的知识转移和理解长指令。

超越已知：研究 LLMs 在域外意图检测中的性能

Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent  Detection

Out-of-Domain (OOD) intent detection is vital for practical dialogue systems,
and it usually requires considering multi-turn dialogue contexts. However, most
previous OOD intent detection approaches are limited to single dialogue turns.
In this paper, we introduce a context-aware OOD intent detection (Caro)
framework to model multi-turn contexts in OOD intent detection tasks.
Specifically, we follow the information bottleneck principle to extract robust
representations from multi-turn dialogue contexts. Two different views are
constructed for each input sample and the superfluous information not related
to intent detection is removed using a multi-view information bottleneck loss.
Moreover, we also explore utilizing unlabeled data in Caro. A two-stage
training process is introduced to mine OOD samples from these unlabeled data,
and these OOD samples are used to train the resulting model with a
bootstrapping approach. Comprehensive experiments demonstrate that Caro
establishes state-of-the-art performances on multi-turn OOD detection tasks by
improving the F1-OOD score of over $29\%$ compared to the previous best method.

本文提出一种上下文感知的 out-of-domain 意图检测（Caro）框架来模拟多轮对话中的上下文，采用信息瓶颈原理从多轮对话上下文中提取鲁棒特征，并通过多视图信息瓶颈损失去除与意图检测无关的冗余信息，在 Caro 中探索了利用未标记的数据，这种方法提升了 29% 以上的 F1-ODD 分数，实验表明 Caro 在多轮意图检测任务中取得了最好的表现。

基于多轮对话上下文的领域外意图检测

Out-of-Domain Intent Detection Considering Multi-turn Dialogue Contexts

Out-of-Domain (OOD) intent detection is important for practical dialog
systems. To alleviate the issue of lacking OOD training samples, some works
propose synthesizing pseudo OOD samples and directly assigning one-hot OOD
labels to these pseudo samples. However, these one-hot labels introduce noises
to the training process because some hard pseudo OOD samples may coincide with
In-Domain (IND) intents. In this paper, we propose an adaptive soft pseudo
labeling (ASoul) method that can estimate soft labels for pseudo OOD samples
when training OOD detectors. Semantic connections between pseudo OOD samples
and IND intents are captured using an embedding graph. A co-training framework
is further introduced to produce resulting soft labels following the smoothness
assumption, i.e., close samples are likely to have similar labels. Extensive
experiments on three benchmark datasets show that ASoul consistently improves
the OOD detection performance and outperforms various competitive baselines.

提出了自适应软伪标签 (ASoul) 方法，用于在训练 OOD 检测器时为伪 OOD 样本估计软标签。该方法通过嵌入图捕获伪 OOD 样本和 IND 意图之间的语义连接，并引入协同训练框架来产生结果软标签。在三个基准数据集上的大量实验证明 ASoul 不断改善 OOD 检测性能并优于各种竞争基线。