To ensure the reliable use of classification systems in medical applications, it is crucial to prevent silent failures. This can be achieved by either designing classifiers that are robust enough to avoid failures in the first place, or by detecting remaining failures using confidence scoring functions (CSFs). A predominant source of failures in image classification is distribution shifts between training data and deployment data. To understand the current state of silent failure prevention in medical imaging, we conduct the first comprehensive analysis comparing various CSFs in four biomedical tasks and a diverse range of distribution shifts. Based on the result that none of the benchmarked CSFs can reliably prevent silent failures, we conclude that a deeper understanding of the root causes of failures in the data is required. To facilitate this, we introduce SF-Visuals, an interactive analysis tool that uses latent space clustering to visualize shifts and failures. On the basis of various examples, we demonstrate how this tool can help researchers gain insight into the requirements for safe application of classification systems in the medical domain. The open-source benchmark and tool are at: https://github.com/IML-DKFZ/sf-visuals.

通过对各种CSFs在四个生物医学任务和多样化的分布转变进行首个综合分析，我们得出结论，没有任何基准CSFs能可靠地防止静默故障，因此需要更深入地了解数据故障的根本原因。为了促进此过程，我们引入了SF-Visuals，一种使用潜在空间聚类来可视化转变和故障的交互式分析工具。通过各种示例，我们展示了该工具如何帮助研究人员洞察在医学领域中安全应用分类系统的要求。

医学图像分类中隐性故障的理解