We consider the task of few-shot intent detection, which involves training a
deep learning model to classify utterances based on their underlying intents
using only a small amount of labeled data. The current approach to address this
problem is through continual pre-training, i.e., fine-tuning pre-trained
language models (PLMs) on external resources (e.g., conversational corpora,
public intent detection datasets, or natural language understanding datasets)
before using them as utterance encoders for training an intent classifier. In
this paper, we show that continual pre-training may not be essential, since the
overfitting problem of PLMs on this task may not be as serious as expected.
Specifically, we find that directly fine-tuning PLMs on only a handful of
labeled examples already yields decent results compared to methods that employ
continual pre-training, and the performance gap diminishes rapidly as the
number of labeled data increases. To maximize the utilization of the limited
available data, we propose a context augmentation method and leverage
sequential self-distillation to boost performance. Comprehensive experiments on
real-world benchmarks show that given only two or more labeled samples per
class, direct fine-tuning outperforms many strong baselines that utilize
external data sources for continual pre-training. The code can be found at
this https URL

本文针对少样本意图检测任务进行研究，提出一种新的方法，即直接在少量标记数据上对预训练语言模型（PLMs）进行微调，使用上下文增强方法和序列自蒸馏技术进一步优化性能。实验表明，该方法优于许多强基线方法，甚至是在只有每类两个或更多标记样本的情况下。