Conformal prediction (CP) is an emerging uncertainty quantification framework
that allows us to construct a prediction set to cover the true label with a
pre-specified marginal or conditional probability. Although the valid coverage
guarantee has been extensively studied for classification problems, CP often
produces large prediction sets which may not be practically useful. This issue
is exacerbated for the setting of class-conditional coverage on imbalanced
classification tasks. This paper proposes the Rank Calibrated Class-conditional
CP (RC3P) algorithm to reduce the prediction set sizes to achieve
class-conditional coverage, where the valid coverage holds for each class. In
contrast to the standard class-conditional CP (CCP) method that uniformly
thresholds the class-wise conformity score for each class, the augmented label
rank calibration step allows RC3P to selectively iterate this class-wise
thresholding subroutine only for a subset of classes whose class-wise top-k
error is small. We prove that agnostic to the classifier and data distribution,
RC3P achieves class-wise coverage. We also show that RC3P reduces the size of
prediction sets compared to the CCP method. Comprehensive experiments on
multiple real-world datasets demonstrate that RC3P achieves class-wise coverage
and 26.25% reduction in prediction set sizes on average.

针对不平衡分类任务的一种新的不确定性量化框架，Rank Calibrated Class-conditional CP (RC3P) 算法，通过降低预测集大小以实现类条件覆盖，对于每个类别都具有有效的覆盖保证。