Progress in graph neural networks has grown rapidly in recent years, with
many new developments in drug discovery, medical diagnosis, and recommender
systems. While this progress is significant, many networks are `black boxes'
with little understanding of the `what' exactly the network is learning. Many
high-stakes applications, such as drug discovery, require human-intelligible
explanations from the models so that users can recognize errors and discover
new knowledge. Therefore, the development of explainable AI algorithms is
essential for us to reap the benefits of AI.
We propose an explainability algorithm for GNNs called eXplainable Insight
(XInsight) that generates a distribution of model explanations using GFlowNets.
Since GFlowNets generate objects with probabilities proportional to a reward,
XInsight can generate a diverse set of explanations, compared to previous
methods that only learn the maximum reward sample. We demonstrate XInsight by
generating explanations for GNNs trained on two graph classification tasks:
classifying mutagenic compounds with the MUTAG dataset and classifying acyclic
graphs with a synthetic dataset that we have open-sourced. We show the utility
of XInsight's explanations by analyzing the generated compounds using QSAR
modeling, and we find that XInsight generates compounds that cluster by
lipophilicity, a known correlate of mutagenicity. Our results show that
XInsight generates a distribution of explanations that uncovers the underlying
relationships demonstrated by the model. They also highlight the importance of
generating a diverse set of explanations, as it enables us to discover hidden
relationships in the model and provides valuable guidance for further analysis.

本研究提出了一种叫做 XInsight 的解释性算法来生成图神经网络的模型解释，以便在高风险应用中（例如药物发现）识别错误并发现新知识。通过对 MUTAG 数据集和一个合成数据集进行实验，证明了 XInsight 的实用性，并且 XInsight 能够生成一组多样化的解释，这有助于发现模型中的隐藏关系并提供有价值的进一步分析指导。