Pre-trained language models (PLMs) have been shown effective for zero-shot
(0shot) text classification. 0shot models based on natural language inference
(NLI) and next sentence prediction (NSP) employ cross-encoder architecture and
infer by making a forward pass through the model for each label-text pair
separately. This increases the computational cost to make inferences linearly
in the number of labels. In this work, we improve the efficiency of such
cross-encoder-based 0shot models by restricting the number of likely labels
using another fast base classifier-based conformal predictor (CP) calibrated on
samples labeled by the 0shot model. Since a CP generates prediction sets with
coverage guarantees, it reduces the number of target labels without excluding
the most probable label based on the 0shot model. We experiment with three
intent and two topic classification datasets. With a suitable CP for each
dataset, we reduce the average inference time for NLI- and NSP-based models by
25.6% and 22.2% respectively, without dropping performance below the predefined
error rate of 1%.

通过使用基于 conformal predictor 的快速基分类器限制潜在的标签数，我们成功地提高了基于交叉编码器的零样本模型的效率，实验结果显示我们可以在性能不降低的情况下，将自然语言推理和下一句话预测的模型推理时间分别降低了 25.6％和 22.2％。

提高零样本文本分类效率的共形预测器

Conformal Predictor for Improving Zero-shot Text Classification Efficiency

Few-shot Intent Classification (FSIC) is one of the key challenges in modular
task-oriented dialog systems. While advanced FSIC methods are similar in using
pretrained language models to encode texts and nearest neighbour-based
inference for classification, these methods differ in details. They start from
different pretrained text encoders, use different encoding architectures with
varying similarity functions, and adopt different training regimes. Coupling
these mostly independent design decisions and the lack of accompanying ablation
studies are big obstacle to identify the factors that drive the reported FSIC
performance. We study these details across three key dimensions: (1) Encoding
architectures: Cross-Encoder vs Bi-Encoders; (2) Similarity function:
Parameterized (i.e., trainable) functions vs non-parameterized function; (3)
Training regimes: Episodic meta-learning vs the straightforward (i.e.,
non-episodic) training. Our experimental results on seven FSIC benchmarks
reveal three important findings. First, the unexplored combination of the
cross-encoder architecture (with parameterized similarity scoring function) and
episodic meta-learning consistently yields the best FSIC performance. Second,
Episodic training yields a more robust FSIC classifier than non-episodic one.
Third, in meta-learning methods, splitting an episode to support and query sets
is not a must. Our findings paves the way for conducting state-of-the-art
research in FSIC and more importantly raise the community's attention to
details of FSIC methods. We release our code and data publicly.

探究几个不同的维度影响最先进的 FSIC 性能，并发现具有参数化相似性计分函数的交叉编码器体系结构与情节元学习的组合可以始终提供最佳的 FSIC 性能。

Few-Shot 意图分类的模型和训练方案问题

The Devil is in the Details: On Models and Training Regimes for Few-Shot Intent Classification

Response retrieval is a subset of neural ranking in which a model selects a
suitable response from a set of candidates given a conversation history.
Retrieval-based chat-bots are typically employed in information seeking
conversational systems such as customer support agents. In order to make
pairwise comparisons between a conversation history and a candidate response,
two approaches are common: cross-encoders performing full self-attention over
the pair and bi-encoders encoding the pair separately. The former gives better
prediction quality but is too slow for practical use. In this paper, we propose
a new cross-encoder architecture and transfer knowledge from this model to a
bi-encoder model using distillation. This effectively boosts bi-encoder
performance at no cost during inference time. We perform a detailed analysis of
this approach on three response retrieval datasets.

本文研究了响应检索中的跨编码器架构及其应用领域，并在此基础上提出了一种新的跨编码器架构，将其应用于双编码器模型中可以大幅提升其检索性能，并在三个响应检索数据集上进行了详细分析。