Prediction sets capture uncertainty by predicting sets of labels rather than individual labels, enabling downstream decisions to conservatively account for all plausible outcomes. Conformal inference algorithms construct prediction sets guaranteed to contain the true label with high probability. These guarantees fail to hold in the face of distribution shift, which is precisely when reliable uncertainty quantification can be most useful. We propose a novel algorithm for constructing prediction sets with PAC guarantees in the label shift setting. This method estimates the predicted probabilities of the classes in a target domain, as well as the confusion matrix, then propagates uncertainty in these estimates through a Gaussian elimination algorithm to compute confidence intervals for importance weights. Finally, it uses these intervals to construct prediction sets. We evaluate our approach on five datasets: the CIFAR-10, ChestX-Ray and Entity-13 image datasets, the tabular CDC Heart dataset, and the AGNews text dataset. Our algorithm satisfies the PAC guarantee while producing smaller, more informative, prediction sets compared to several baselines.

使用一种新颖的算法，在标签偏移情境下构建具有PAC保证的预测集合，通过对目标领域的类别的预测概率和混淆矩阵进行估计，利用高斯消元算法传播这些估计的不确定性，计算重要性权重的置信区间，并利用这些区间构建预测集合。在几个基准数据集上评估我们的方法，证明该算法满足PAC保证，并且相较于几个基准算法，生成更小且更具信息量的预测集合。

标签转移下的PAC预测集