The optimal prediction strategy for out-of-distribution (OOD) setups is a
fundamental question in machine learning. In this paper, we address this
question and present several contributions. We propose three reject option
models for OOD setups: the Cost-based model, the Bounded TPR-FPR model, and the
Bounded Precision-Recall model. These models extend the standard reject option
models used in non-OOD setups and define the notion of an optimal OOD selective
classifier. We establish that all the proposed models, despite their different
formulations, share a common class of optimal strategies. Motivated by the
optimal strategy, we introduce double-score OOD methods that leverage
uncertainty scores from two chosen OOD detectors: one focused on OOD/ID
discrimination and the other on misclassification detection. The experimental
results consistently demonstrate the superior performance of this simple
strategy compared to state-of-the-art methods. Additionally, we propose novel
evaluation metrics derived from the definition of the optimal strategy under
the proposed OOD rejection models. These new metrics provide a comprehensive
and reliable assessment of OOD methods without the deficiencies observed in
existing evaluation approaches.

本研究提出了三种拒绝选项模型和双分数 OOD 方法，以及与之配套的新评估指标，它们能够显著提高除错分类和未知样本特征识别任务中的性能。