Saliency maps can explain how deep neural networks classify images. But are
they actually useful for humans? The present systematic review of 68 user
studies found that while saliency maps can enhance human performance, null
effects or even costs are quite common. To investigate what modulates these
effects, the empirical outcomes were organised along several factors related to
the human tasks, AI performance, XAI methods, images to be classified, human
participants and comparison conditions. In image-focused tasks, benefits were
less common than in AI-focused tasks, but the effects depended on the specific
cognitive requirements. Moreover, benefits were usually restricted to incorrect
AI predictions in AI-focused tasks but to correct ones in image-focused tasks.
XAI-related factors had surprisingly little impact. The evidence was limited
for image- and human-related factors and the effects were highly dependent on
the comparison conditions. These findings may support the design of future user
studies.

68 个用户研究的系统综述发现，显著性图可以增强人类表现，但也常见到零效应甚至负效应。这些效应受多个因素的调节，包括人类任务、AI 性能、XAI 方法、待分类的图像、人类参与者和比较条件等。图像任务中的效益不如 AI 任务中常见，但效果取决于具体的认知需求。此外，效益通常局限于 AI 任务中的错误预测，而在图像任务中局限于正确预测。与 XAI 相关的因素影响较小。对于图像和人类相关的因素，证据有限且效应高度依赖于比较条件。这些发现可能有助于未来用户研究的设计。