The success of deep learning models deployed in the real world depends
critically on their ability to generalize well across diverse data domains.
Here, we address a fundamental challenge with selective classification during
automated diagnosis with domain-shifted medical images. In this scenario,
models must learn to avoid making predictions when label confidence is low,
especially when tested with samples far removed from the training set
(covariate shift). Such uncertain cases are typically referred to the clinician
for further analysis and evaluation. Yet, we show that even state-of-the-art
domain generalization approaches fail severely during referral when tested on
medical images acquired from a different demographic or using a different
technology. We examine two benchmark diagnostic medical imaging datasets
exhibiting strong covariate shifts: i) diabetic retinopathy prediction with
retinal fundus images and ii) multilabel disease prediction with chest X-ray
images. We show that predictive uncertainty estimates do not generalize well
under covariate shifts leading to non-monotonic referral curves, and severe
drops in performance (up to 50%) at high referral rates (>70%). We evaluate
novel combinations of robust generalization and post hoc referral approaches,
that rescue these failures and achieve significant performance improvements,
typically >10%, over baseline methods. Our study identifies a critical
challenge with referral in domain-shifted medical images and finds key
applications in reliable, automated disease diagnosis.

对领域转变的医学图像中的可靠自动疾病诊断中的转诊存在重要挑战，并找到了关键应用。