Shifts in data distribution can substantially harm the performance of clinical AI models. Hence, various methods have been developed to detect the presence of such shifts at deployment time. However, root causes of dataset shifts are varied, and the choice of shift mitigation strategies is highly dependent on the precise type of shift encountered at test time. As such, detecting test-time dataset shift is not sufficient: precisely identifying which type of shift has occurred is critical. In this work, we propose the first unsupervised dataset shift identification framework, effectively distinguishing between prevalence shift (caused by a change in the label distribution), covariate shift (caused by a change in input characteristics) and mixed shifts (simultaneous prevalence and covariate shifts). We discuss the importance of self-supervised encoders for detecting subtle covariate shifts and propose a novel shift detector leveraging both self-supervised encoders and task model outputs for improved shift detection. We report promising results for the proposed shift identification framework across three different imaging modalities (chest radiography, digital mammography, and retinal fundus images) on five types of real-world dataset shifts, using four large publicly available datasets.

本研究针对临床AI模型在数据分布变化时性能显著下降的问题，提出了一种首个无监督数据集偏移识别框架，能够有效区分标签分布变化引起的流行偏移、输入特征变化导致的协变量偏移以及两者同时存在的混合偏移。研究表明，利用自监督编码器和任务模型输出结合的新型偏移检测器在三种不同影像模式下表现出色，能够促进对现实世界中五种数据集偏移类型的检测。 

自动数据集偏移识别以支持AI性能漂移的根本原因分析