Classification with rejection emerges as a learning paradigm which allows
models to abstain from making predictions. The predominant approach is to alter
the supervised learning pipeline by augmenting typical loss functions, letting
model rejection incur a lower loss than an incorrect prediction. Instead, we
propose a different distributional perspective, where we seek to find an
idealized data distribution which maximizes a pretrained model's performance.
This can be formalized via the optimization of a loss's risk with a $
\phi$-divergence regularization term. Through this idealized distribution, a
rejection decision can be made by utilizing the density ratio between this
distribution and the data distribution. We focus on the setting where our $
\phi $-divergences are specified by the family of $ \alpha $-divergence. Our
framework is tested empirically over clean and noisy datasets.

通过改变监督学习流程和优化损失函数的风险，提出了一种以拒绝为基础的学习范式，通过利用密度比率来决定模型的拒绝决策，并在干净和噪声数据集上进行了实证测试。

通过学习密度比进行拒绝

Rejection via Learning Density Ratios

Both transduction and rejection have emerged as important techniques for
defending against adversarial perturbations. A recent work by Tram\`er showed
that, in the rejection-only case (no transduction), a strong rejection-solution
can be turned into a strong (but computationally inefficient) non-rejection
solution. This detector-to-classifier reduction has been mostly applied to give
evidence that certain claims of strong selective-model solutions are
susceptible, leaving the benefits of rejection unclear. On the other hand, a
recent work by Goldwasser et al. showed that rejection combined with
transduction can give provable guarantees (for certain problems) that cannot be
achieved otherwise. Nevertheless, under recent strong adversarial attacks
(GMSA, which has been shown to be much more effective than AutoAttack against
transduction), Goldwasser et al.'s work was shown to have low performance in a
practical deep-learning setting. In this paper, we take a step towards
realizing the promise of transduction+rejection in more realistic scenarios.
Theoretically, we show that a novel application of Tram\`er's
classifier-to-detector technique in the transductive setting can give
significantly improved sample-complexity for robust generalization. While our
theoretical construction is computationally inefficient, it guides us to
identify an efficient transductive algorithm to learn a selective model.
Extensive experiments using state of the art attacks (AutoAttack, GMSA) show
that our solutions provide significantly better robust accuracy.

在拒绝和转导两个技术中，拒绝 + 转导可提供显著改进，本研究发展了一种新的转导方法，结合 Tram'er 的分类器 - 探测器技术，达到更好的鲁棒性。

双管齐下：通过结合转导和拒绝机制实现更好的对抗鲁棒性

Two Heads are Better than One: Towards Better Adversarial Robustness by  Combining Transduction and Rejection

We investigate the problem of multiclass classification with rejection, where
a classifier can choose not to make a prediction to avoid critical
misclassification. First, we consider an approach based on simultaneous
training of a classifier and a rejector, which achieves the state-of-the-art
performance in the binary case. We analyze this approach for the multiclass
case and derive a general condition for calibration to the Bayes-optimal
solution, which suggests that calibration is hard to achieve by general loss
functions unlike the binary case. Next, we consider another traditional
approach based on confidence scores, in which the existing work focuses on a
specific class of losses. We propose rejection criteria for more general losses
for this approach and guarantee calibration to the Bayes-optimal solution.
Finally, we conduct experiments to validate the relevance of our theoretical
findings.

本文研究了多类别分类问题中的拒绝机制，提出了同时训练分类器和拒绝器的方法，并探讨了针对更广泛损失函数的可接受标准，最终通过实验验证了理论发现的相关性。