The field of explainable artificial intelligence (XAI) aims to uncover the
inner workings of complex deep neural models. While being crucial for
safety-critical domains, XAI inherently lacks ground-truth explanations, making
its automatic evaluation an unsolved problem. We address this challenge by
proposing a novel synthetic vision dataset, named FunnyBirds, and accompanying
automatic evaluation protocols. Our dataset allows performing semantically
meaningful image interventions, e.g., removing individual object parts, which
has three important implications. First, it enables analyzing explanations on a
part level, which is closer to human comprehension than existing methods that
evaluate on a pixel level. Second, by comparing the model output for inputs
with removed parts, we can estimate ground-truth part importances that should
be reflected in the explanations. Third, by mapping individual explanations
into a common space of part importances, we can analyze a variety of different
explanation types in a single common framework. Using our tools, we report
results for 24 different combinations of neural models and XAI methods,
demonstrating the strengths and weaknesses of the assessed methods in a fully
automatic and systematic manner.

通过使用一个名为 FunnyBirds 的新型合成视觉数据集以及相应的自动评估协议，我们解决了解释性人工智能（XAI）领域中的一个挑战，该领域旨在揭示复杂深度神经模型的内部工作方式。使用我们的工具，我们对 24 种不同的神经模型和 XAI 方法进行了评估，以全自动和系统的方式展示了这些方法的优势和劣势。