Semantic segmentation is an essential component of medical image analysis
research, with recent deep learning algorithms offering out-of-the-box
applicability across diverse datasets. Despite these advancements, segmentation
failures remain a significant concern for real-world clinical applications,
necessitating reliable detection mechanisms. This paper introduces a
comprehensive benchmarking framework aimed at evaluating failure detection
methodologies within medical image segmentation. Through our analysis, we
identify the strengths and limitations of current failure detection metrics,
advocating for the risk-coverage analysis as a holistic evaluation approach.
Utilizing a collective dataset comprising five public 3D medical image
collections, we assess the efficacy of various failure detection strategies
under realistic test-time distribution shifts. Our findings highlight the
importance of pixel confidence aggregation and we observe superior performance
of the pairwise Dice score (Roy et al., 2019) between ensemble predictions,
positioning it as a simple and robust baseline for failure detection in medical
image segmentation. To promote ongoing research, we make the benchmarking
framework available to the community.

本文介绍了一个用于评估医学图像分割中失败检测方法的综合基准框架，通过分析我们确定了当前失败检测度量的优点和局限性，并推崇风险覆盖分析作为一种整体评估方法，通过在包含五个公开的三维医学图像集合的综合数据集上评估各种失败检测策略的有效性，我们发现像素置信度聚合的重要性，并且观察到在医学图像分割中，集合预测之间的成对 Dice 分数（Roy 等人，2019）表现优秀，将其定位为简单而稳健的失败检测基准。为了促进持续的研究，我们将基准框架提供给社区使用。