Downstream applications often require text classification models to be
accurate, robust, and interpretable. While the accuracy of the stateof-the-art
language models approximates human performance, they are not designed to be
interpretable and often exhibit a drop in performance on noisy data. The family
of PrototypeBased Networks (PBNs) that classify examples based on their
similarity to prototypical examples of a class (prototypes) is natively
interpretable and shown to be robust to noise, which enabled its wide usage for
computer vision tasks. In this paper, we study whether the robustness
properties of PBNs transfer to text classification tasks. We design a modular
and comprehensive framework for studying PBNs, which includes different
backbone architectures, backbone sizes, and objective functions. Our evaluation
protocol assesses the robustness of models against character-, word-, and
sentence-level perturbations. Our experiments on three benchmarks show that the
robustness of PBNs transfers to NLP classification tasks facing realistic
perturbations. Moreover, the robustness of PBNs is supported mostly by the
objective function that keeps prototypes interpretable, while the robustness
superiority of PBNs over vanilla models becomes more salient as datasets get
more complex.

基于原型网络 (Prototype-Based Networks) 的鲁棒性在自然语言处理 (NLP) 文本分类任务中能够面对现实扰动，并且主要由保持原型可解释性的目标函数支持。

鲁棒文本分类：分析基于原型的网络

Robust Text Classification: Analyzing Prototype-Based Networks

Distance Metric Learning (DML) has attracted much attention in image
processing in recent years. This paper analyzes its impact on supervised
fine-tuning language models for Natural Language Processing (NLP)
classification tasks under few-shot learning settings. We investigated several
DML loss functions in training RoBERTa language models on known SentEval
Transfer Tasks datasets. We also analyzed the possibility of using proxy-based
DML losses during model inference.
Our systematic experiments have shown that under few-shot learning settings,
particularly proxy-based DML losses can positively affect the fine-tuning and
inference of a supervised language model. Models tuned with a combination of
CCE (categorical cross-entropy loss) and ProxyAnchor Loss have, on average, the
best performance and outperform models with only CCE by about 3.27 percentage
points -- up to 10.38 percentage points depending on the training dataset.

本论文研究了距离度量学习 (Distance Metric Learning) 对少样本学习下的监督学习语言模型 Fine-tuning 在自然语言处理 (NLP) 分类任务中的影响，并发现使用基于代理的 DML 损失有助于模型的精调和推理，尤其是使用 CCE（分类交叉熵）和 ProxyAnchor Loss 的结合模型，其平均性能优于仅使用 CCE 的模型，提高了 3.27 个百分点 -- 具体取决于训练数据集的不同。

重新审视距离度量学习在少样本自然语言分类中的应用

Revisiting Distance Metric Learning for Few-Shot Natural Language Classification

With the rapid development in deep learning, deep neural networks have been
widely adopted in many real-life natural language applications. Under deep
neural networks, a pre-defined vocabulary is required to vectorize text inputs.
The canonical approach to select pre-defined vocabulary is based on the word
frequency, where a threshold is selected to cut off the long tail distribution.
However, we observed that such simple approach could easily lead to under-sized
vocabulary or over-sized vocabulary issues. Therefore, we are interested in
understanding how the end-task classification accuracy is related to the
vocabulary size and what is the minimum required vocabulary size to achieve a
specific performance. In this paper, we provide a more sophisticated
variational vocabulary dropout (VVD) based on variational dropout to perform
vocabulary selection, which can intelligently select the subset of the
vocabulary to achieve the required performance. To evaluate different
algorithms on the newly proposed vocabulary selection problem, we propose two
new metrics: Area Under Accuracy-Vocab Curve and Vocab Size under X\% Accuracy
Drop. Through extensive experiments on various NLP classification tasks, our
variational framework is shown to significantly outperform the frequency-based
and other selection baselines on these metrics.

本论文提出了一种基于变分 dropout 的变分词汇 dropout（VVD）方法，用于智能地选择词汇子集以达到所需性能，通过在多个 NLP 分类任务上进行广泛实验证明了该方法在语料库选择问题上显著优于基于词频的和其他选择基准。