Due to the black-box nature of deep learning models, methods for explaining
the models' results are crucial to gain trust from humans and support
collaboration between AIs and humans. In this paper, we consider several
model-agnostic and model-specific explanation methods for CNNs for text
classification and conduct three human-grounded evaluations, focusing on
different purposes of explanations: (1) revealing model behavior, (2)
justifying model predictions, and (3) helping humans investigate uncertain
predictions. The results highlight dissimilar qualities of the various
explanation methods we consider and show the degree to which these methods
could serve for each purpose.

本文探讨了一些适用于 CNN 文本分类的模型无关和模型特定解释方法，并进行了三次人类评估来揭示这些方法揭示模型行为、证明模型预测和帮助人类调查不确定预测的不同效果。结果表明，这些方法具有不同的性质，并且可以为每个目的提供帮助。

基于人工标注的文本分类解释方法评估

Human-grounded Evaluations of Explanation Methods for Text  Classification

Privacy and transparency are two key foundations of trustworthy machine
learning. Model explanations offer insights into a model's decisions on input
data, whereas privacy is primarily concerned with protecting information about
the training data. We analyze connections between model explanations and the
leakage of sensitive information about the model's training set. We investigate
the privacy risks of feature-based model explanations using membership
inference attacks: quantifying how much model predictions plus their
explanations leak information about the presence of a datapoint in the training
set of a model. We extensively evaluate membership inference attacks based on
feature-based model explanations, over a variety of datasets. We show that
backpropagation-based explanations can leak a significant amount of information
about individual training datapoints. This is because they reveal statistical
information about the decision boundaries of the model about an input, which
can reveal its membership. We also empirically investigate the trade-off
between privacy and explanation quality, by studying the perturbation-based
model explanations.

本研究分析了模型解释和训练数据隐私保护之间的联系，通过特征模型解释的成员推断攻击评估了隐私风险，并研究了隐私和解释质量之间的平衡。

模型解释的隐私风险

On the Privacy Risks of Model Explanations

The ubiquity of machine learning based predictive models in modern society
naturally leads people to ask how trustworthy those models are? In predictive
modeling, it is quite common to induce a trade-off between accuracy and
interpretability. For instance, doctors would like to know how effective some
treatment will be for a patient or why the model suggested a particular
medication for a patient exhibiting those symptoms? We acknowledge that the
necessity for interpretability is a consequence of an incomplete formalisation
of the problem, or more precisely of multiple meanings adhered to a particular
concept. For certain problems, it is not enough to get the answer (what), the
model also has to provide an explanation of how it came to that conclusion
(why), because a correct prediction, only partially solves the original
problem. In this article we extend existing categorisation of techniques to aid
model interpretability and test this categorisation.

本文研究机器学习预测模型的可靠性和解释性的平衡，并提出解释模型的技术分类并测试其有效性。

基于后验解释的预测模型分类

A Categorisation of Post-hoc Explanations for Predictive Models

Understanding and interpreting how machine learning (ML) models make
decisions have been a big challenge. While recent research has proposed various
technical approaches to provide some clues as to how an ML model makes
individual predictions, they cannot provide users with an ability to inspect a
model as a complete entity. In this work, we propose a novel technical approach
that augments a Bayesian non-parametric regression mixture model with multiple
elastic nets. Using the enhanced mixture model, we can extract generalizable
insights for a target model through a global approximation. To demonstrate the
utility of our approach, we evaluate it on different ML models in the context
of image recognition. The empirical results indicate that our proposed approach
not only outperforms the state-of-the-art techniques in explaining individual
decisions but also provides users with an ability to discover the
vulnerabilities of the target ML models.

本研究提出了一种新的技术方法，将贝叶斯非参数回归混合模型与多个弹性网络相结合，从而提取目标模型的可推广见解。通过在图像识别的上下文中对不同的机器学习模型进行评估，实验结果表明，我们提出的方法不仅在解释个别决策方面优于现有技术，而且为用户提供了发现目标机器学习模型漏洞的能力。