The tasks of out-of-domain (OOD) intent discovery and generalized intent
discovery (GID) aim to extend a closed intent classifier to open-world intent
sets, which is crucial to task-oriented dialogue (TOD) systems. Previous
methods address them by fine-tuning discriminative models. Recently, although
some studies have been exploring the application of large language models
(LLMs) represented by ChatGPT to various downstream tasks, it is still unclear
for the ability of ChatGPT to discover and incrementally extent OOD intents. In
this paper, we comprehensively evaluate ChatGPT on OOD intent discovery and
GID, and then outline the strengths and weaknesses of ChatGPT. Overall, ChatGPT
exhibits consistent advantages under zero-shot settings, but is still at a
disadvantage compared to fine-tuned models. More deeply, through a series of
analytical experiments, we summarize and discuss the challenges faced by LLMs
including clustering, domain-specific understanding, and cross-domain
in-context learning scenarios. Finally, we provide empirical guidance for
future directions to address these challenges.

ChatGPT 对 OOD 意图探索和广义意图探索进行了全面评估，并概述了 ChatGPT 的优势和劣势。ChatGPT 在零样本设置下展现了一致的优势，但与微调模型相比仍处于劣势。通过一系列分析实验，我们总结和讨论了 LLM 面临的挑战，包括聚类、领域特定理解和跨领域情境学习场景。最后，我们提供了未来解决这些挑战的经验指导。