Our Visual Analytics (VA) tool ScrutinAI supports human analysts to investigate interactively model performanceand data sets. Model performance depends on labeling quality to a large extent. In particular in medical settings, generation of high quality labels requires in depth expert knowledge and is very costly. Often, data sets are labeled by collecting opinions of groups of experts. We use our VA tool to analyse the influence of label variations between different experts on the model performance. ScrutinAI facilitates to perform a root cause analysis that distinguishes weaknesses of deep neural network (DNN) models caused by varying or missing labeling quality from true weaknesses. We scrutinize the overall detection of intracranial hemorrhages and the more subtle differentiation between subtypes in a publicly available data set.

我们的可视化分析工具ScrutinAI支持人工分析师与模型性能和数据集进行互动研究。通过使用我们的可视化工具，我们分析了不同专家之间标签变异对模型性能的影响，并能够区分来自标签质量的变化或缺失的深度神经网络模型的缺陷和真实缺陷，以便在公开可用的数据集中全面检测颅内出血和不同类型之间的细微差别。

在医疗应用案例中使用ScrutinAI进行深度神经网络性能的视觉检查