Few-shot intent classification and slot filling are important but challenging
tasks due to the scarcity of finely labeled data. Therefore, current works
first train a model on source domains with sufficiently labeled data, and then
transfer the model to target domains where only rarely labeled data is
available. However, experience transferring as a whole usually suffers from
gaps that exist among source domains and target domains. For instance,
transferring domain-specific-knowledge-related experience is difficult. To
tackle this problem, we propose a new method that explicitly decouples the
transferring of general-semantic-representation-related experience and the
domain-specific-knowledge-related experience. Specifically, for
domain-specific-knowledge-related experience, we design two modules to capture
intent-slot relation and slot-slot relation respectively. Extensive experiments
on Snips and FewJoint datasets show that our method achieves state-of-the-art
performance. The method improves the joint accuracy metric from 27.72% to
42.20% in the 1-shot setting, and from 46.54% to 60.79% in the 5-shot setting.

通过解耦普遍语义表示相关与特定领域知识相关的经验，将 Few-shot 意图分类和槽填充任务中的域特定知识传递与其他经验分开，显著提升性能。

少样本意图分类与槽位填充的表示和知识分离

Decoupling Representation and Knowledge for Few-Shot Intent  Classification and Slot Filling

Few-shot Intent Classification (FSIC) is one of the key challenges in modular
task-oriented dialog systems. While advanced FSIC methods are similar in using
pretrained language models to encode texts and nearest neighbour-based
inference for classification, these methods differ in details. They start from
different pretrained text encoders, use different encoding architectures with
varying similarity functions, and adopt different training regimes. Coupling
these mostly independent design decisions and the lack of accompanying ablation
studies are big obstacle to identify the factors that drive the reported FSIC
performance. We study these details across three key dimensions: (1) Encoding
architectures: Cross-Encoder vs Bi-Encoders; (2) Similarity function:
Parameterized (i.e., trainable) functions vs non-parameterized function; (3)
Training regimes: Episodic meta-learning vs the straightforward (i.e.,
non-episodic) training. Our experimental results on seven FSIC benchmarks
reveal three important findings. First, the unexplored combination of the
cross-encoder architecture (with parameterized similarity scoring function) and
episodic meta-learning consistently yields the best FSIC performance. Second,
Episodic training yields a more robust FSIC classifier than non-episodic one.
Third, in meta-learning methods, splitting an episode to support and query sets
is not a must. Our findings paves the way for conducting state-of-the-art
research in FSIC and more importantly raise the community's attention to
details of FSIC methods. We release our code and data publicly.

探究几个不同的维度影响最先进的 FSIC 性能，并发现具有参数化相似性计分函数的交叉编码器体系结构与情节元学习的组合可以始终提供最佳的 FSIC 性能。

Few-Shot 意图分类的模型和训练方案问题

The Devil is in the Details: On Models and Training Regimes for Few-Shot Intent Classification

This paper investigates the effectiveness of pre-training for few-shot intent
classification. While existing paradigms commonly further pre-train language
models such as BERT on a vast amount of unlabeled corpus, we find it highly
effective and efficient to simply fine-tune BERT with a small set of labeled
utterances from public datasets. Specifically, fine-tuning BERT with roughly
1,000 labeled data yields a pre-trained model -- IntentBERT, which can easily
surpass the performance of existing pre-trained models for few-shot intent
classification on novel domains with very different semantics. The high
effectiveness of IntentBERT confirms the feasibility and practicality of
few-shot intent detection, and its high generalization ability across different
domains suggests that intent classification tasks may share a similar
underlying structure, which can be efficiently learned from a small set of
labeled data. The source code can be found at
this https URL

本文研究了少样本意图分类预训练的有效性，发现仅使用公共数据集上的少量标记数据对 BERT 进行微调即可高效地生成一个预训练模型 IntentBERT，其性能超过现有预训练模型，具有高泛化能力。