The black box nature of deep learning models complicate their usage in critical applications such as remote sensing. Conformal prediction is a method to ensure trust in such scenarios. Subject to data exchangeability, conformal prediction provides finite sample coverage guarantees in the form of a prediction set that is guaranteed to contain the true class within a user defined error rate. In this letter we show that conformal prediction algorithms are related to the uncertainty of the deep learning model and that this relation can be used to detect if the deep learning model is out-of-calibration. Popular classification models like Resnet50, Densenet161, InceptionV3, and MobileNetV2 are applied on remote sensing datasets such as the EuroSAT to demonstrate how under noisy scenarios the model outputs become untrustworthy. Furthermore an out-of-calibration detection procedure relating the model uncertainty and the average size of the conformal prediction set is presented.

在遥感等关键应用中，深度学习模型的黑盒特性使其使用变得复杂。与数据可交换性相关的符合预测是一种能确保信任的方法，可提供有限样本覆盖保证，以用户定义的错误率为形式，确保预测集合包含真实类别。本文展示了符合预测算法与深度学习模型的不确定性相关，并可用于检测模型是否未校准。通过将Resnet50、Densenet161、InceptionV3和MobileNetV2等常见分类模型应用于遥感数据集，如EuroSAT，证明在噪声场景下模型的输出变得不可靠。此外，还提出了一种基于模型不确定性和符合预测集合平均大小的未校准检测程序。

使用符合预测法进行深度学习模型校准外检测