For real-world language applications, detecting an out-of-distribution (OOD)
sample is helpful to alert users or reject such unreliable samples. However,
modern over-parameterized language models often produce overconfident
predictions for both in-distribution (ID) and OOD samples. In particular,
language models suffer from OOD samples with a similar semantic representation
to ID samples since these OOD samples lie near the ID manifold. A rejection
network can be trained with ID and diverse outlier samples to detect test OOD
samples, but explicitly collecting auxiliary OOD datasets brings an additional
burden for data collection. In this paper, we propose a simple but effective
method called Pseudo Outlier Exposure (POE) that constructs a surrogate OOD
dataset by sequentially masking tokens related to ID classes. The surrogate OOD
sample introduced by POE shows a similar representation to ID data, which is
most effective in training a rejection network. Our method does not require any
external OOD data and can be easily implemented within off-the-shelf
Transformers. A comprehensive comparison with state-of-the-art algorithms
demonstrates POE's competitiveness on several text classification benchmarks.

我们提出了一种名为 Pseudo Outlier Exposure（POE）的简单但有效的方法，通过顺序遮盖与 ID 类相关的令牌来构建一个替代性的 OOD 数据集，该替代性 OOD 样本与 ID 数据具有相似的表示，可以在训练拒绝网络方面发挥最有效的作用，与最先进的算法进行全面比较，在几个文本分类基准测试中证明了 POE 的竞争力。

使用预训练转换器的伪异常值暴露进行对外分布检测

Pseudo Outlier Exposure for Out-of-Distribution Detection using  Pretrained Transformers

The conventional success of textual classification relies on annotated data,
and the new paradigm of pre-trained language models (PLMs) still requires a few
labeled data for downstream tasks. However, in real-world applications, label
noise inevitably exists in training data, damaging the effectiveness,
robustness, and generalization of the models constructed on such data.
Recently, remarkable achievements have been made to mitigate this dilemma in
visual data, while only a few explore textual data. To fill this gap, we
present SelfMix, a simple yet effective method, to handle label noise in text
classification tasks. SelfMix uses the Gaussian Mixture Model to separate
samples and leverages semi-supervised learning. Unlike previous works requiring
multiple models, our method utilizes the dropout mechanism on a single model to
reduce the confirmation bias in self-training and introduces a textual-level
mixup training strategy. Experimental results on three text classification
benchmarks with different types of text show that the performance of our
proposed method outperforms these strong baselines designed for both textual
and visual data under different noise ratios and noise types. Our code is
available at this https URL.

本文提出一种用于处理文本分类任务中标签噪声的简单而有效的方法 SelfMix，该方法使用高斯混合模型来分离样本，并利用半监督学习。实验结果表明，我们的方法在不同类型的文本噪声下，比设计用于文本和视觉数据的强基线表现更优秀。