Machine learning classification tasks often benefit from predicting a set of possible labels with confidence scores to capture uncertainty. However, existing methods struggle with the high-dimensional nature of the data and the lack of well-calibrated probabilities from modern classification models. We propose a novel conformal prediction method that employs a rank-based score function suitable for classification models that predict the order of labels correctly, even if not well-calibrated. Our approach constructs prediction sets that achieve the desired coverage rate while managing their size. We provide a theoretical analysis of the expected size of the conformal prediction sets based on the rank distribution of the underlying classifier. Through extensive experiments, we demonstrate that our method outperforms existing techniques on various datasets, providing reliable uncertainty quantification. Our contributions include a novel conformal prediction method, theoretical analysis, and empirical evaluation. This work advances the practical deployment of machine learning systems by enabling reliable uncertainty quantification.

通过使用适用于对标签顺序进行正确预测的分类模型的排名式评分函数，我们提出了一种新颖的符合性预测方法，该方法能够预测一组可能的标签，并获得相应的置信度得分以捕捉不确定性。我们的方法构建了在实现期望覆盖率的同时管理其大小的预测集合，通过对底层分类器的排名分布进行理论分析，我们提供了符合性预测集合的预期大小的理论分析。通过广泛的实验证明了我们的方法在各种数据集上优于现有技术，提供了可靠的不确定性量化。本文的贡献包括一种新颖的符合性预测方法、理论分析和实证评估，通过实现可靠的不确定性量化，推动了机器学习系统的实际部署。

基于排名的符合预测集的可信分类