In the past years, many new explanation methods have been proposed to achieve
interpretability of machine learning predictions. However, the utility of these
methods in practical applications has not been researched extensively. In this
paper we present the results of a human-grounded evaluation of SHAP, an
explanation method that has been well-received in the XAI and related
communities. In particular, we study whether this local model-agnostic
explanation method can be useful for real human domain experts to assess the
correctness of positive predictions, i.e. alerts generated by a classifier. We
performed experimentation with three different groups of participants (159 in
total), who had basic knowledge of explainable machine learning. We performed a
qualitative analysis of recorded reflections of experiment participants
performing alert processing with and without SHAP information. The results
suggest that the SHAP explanations do impact the decision-making process,
although the model's confidence score remains to be a leading source of
evidence. We statistically test whether there is a significant difference in
task utility metrics between tasks for which an explanation was available and
tasks in which it was not provided. As opposed to common intuitions, we did not
find a significant difference in alert processing performance when a SHAP
explanation is available compared to when it is not.

在本研究中，我们对一种名为 SHAP 的解释方法进行人本评估，该方法已在可解释人工智能和相关社区中广受欢迎。我们研究了这种局部规范无关解释方法是否对实际的人类领域专家有用，以评估分类器产生的正面预测的正确性。结果表明，SHAP 解释确实影响决策过程，尽管该方法得分的置信度仍然是主要的证据来源。但在我们的实验中，当提示信息可用时，与不提供提示信息相比，在警报处理性能方面并没有明显差异。