Interpretability of artificial intelligence (AI) methods, particularly deep neural networks, is of great interest due to the widespread use of AI-backed systems, which often have unexplainable behavior. The interpretability of such models is a crucial component of building trusted systems. Many methods exist to approach this problem, but they do not obviously generalize to the quantum setting. Here we explore the interpretability of quantum neural networks using local model-agnostic interpretability measures of quantum and classical neural networks. We introduce the concept of the band of inexplicability, representing the interpretable region in which data samples have no explanation, likely victims of inherently random quantum measurements. We see this as a step toward understanding how to build responsible and accountable quantum AI models.

人工智能（AI）方法的可解释性，尤其是深度神经网络的可解释性，引起了广泛关注，因为AI支持的系统往往具有无法解释的行为。本文探讨了量子神经网络的可解释性，并使用量子和经典神经网络的本地模型无关解释性度量方法。我们引入了“不可解释性带”的概念，表示数据样本在该区域没有解释，很可能成为量子随机测量的受害者。我们将此视为向理解如何构建负责任和可追溯的量子AI模型迈出的一步。

量子神经网络的解释性与不可解释性