Despite the recent progress in deep neural networks (DNNs), it remains
challenging to explain the predictions made by DNNs. Existing explanation
methods for DNNs mainly focus on post-hoc explanations where another
explanatory model is employed to provide explanations. The fact that post-hoc
methods can fail to reveal the actual original reasoning process of DNNs raises
the need to build DNNs with built-in interpretability. Motivated by this, many
self-explaining neural networks have been proposed to generate not only
accurate predictions but also clear and intuitive insights into why a
particular decision was made. However, existing self-explaining networks are
limited in providing distribution-free uncertainty quantification for the two
simultaneously generated prediction outcomes (i.e., a sample's final prediction
and its corresponding explanations for interpreting that prediction).
Importantly, they also fail to establish a connection between the confidence
values assigned to the generated explanations in the interpretation layer and
those allocated to the final predictions in the ultimate prediction layer. To
tackle the aforementioned challenges, in this paper, we design a novel
uncertainty modeling framework for self-explaining networks, which not only
demonstrates strong distribution-free uncertainty modeling performance for the
generated explanations in the interpretation layer but also excels in producing
efficient and effective prediction sets for the final predictions based on the
informative high-level basis explanations. We perform the theoretical analysis
for the proposed framework. Extensive experimental evaluation demonstrates the
effectiveness of the proposed uncertainty framework.

通过设计一种新颖的不确定性建模框架，本研究旨在解决深度神经网络在解释性方面的挑战，实现分布无关的不确定性建模，并在生成的解释层中分配与最终预测分配给予的信心值间建立联系，同时在高级解释基础上产生高效和有效的预测集。理论分析和大量的实验评估证明了该不确定性框架的有效性。

基于可信区间预测的自解释神经网络不确定性建模

Towards Modeling Uncertainties of Self-explaining Neural Networks via  Conformal Prediction

Explanations in Computer Vision are often desired, but most Deep Neural
Networks can only provide saliency maps with questionable faithfulness.
Self-Explaining Neural Networks (SENN) extract interpretable concepts with
fidelity, diversity, and grounding to combine them linearly for
decision-making. While they can explain what was recognized, initial
realizations lack accuracy and general applicability. We propose the
Quantized-Self-Explaining Neural Network Q-SENN. Q-SENN satisfies or exceeds
the desiderata of SENN while being applicable to more complex datasets and
maintaining most or all of the accuracy of an uninterpretable baseline model,
out-performing previous work in all considered metrics. Q-SENN describes the
relationship between every class and feature as either positive, negative or
neutral instead of an arbitrary number of possible relations, enforcing more
binary human-friendly features. Since every class is assigned just 5
interpretable features on average, Q-SENN shows convincing local and global
interpretability. Additionally, we propose a feature alignment method, capable
of aligning learned features with human language-based concepts without
additional supervision. Thus, what is learned can be more easily verbalized.
The code is published: this https URL

计算机视觉中的解释通常是需要的，但大多数深度神经网络只能提供有问题的准确性的显著性图。自说明神经网络（SENN）提取可解释的概念，并将它们线性组合用于决策。我们提出了量化自说明神经网络 Q-SENN，它能够满足或超越 SENN 的期望，并适用于更复杂的数据集，并保持不可解释基准模型的准确性，以所有考虑的指标来看，优于以往的工作。Q-SENN 将每个类别与特征之间的关系描述为正、负或中性，而不是一些任意可能的关系，从而实现了更二元的人类友好特征。由于每个类别平均只分配了 5 个可解释特征，Q-SENN 具有令人信服的局部和全局可解释性。此外，我们提出了一种特征对齐方法，能够在无需额外监督的情况下将学到的特征与基于人类语言的概念对齐。因此，所学到的内容可以更容易地用语言表达出来。

Q-SENN：量化自解释神经网络

Q-SENN: Quantized Self-Explaining Neural Networks

Rising usage of deep neural networks to perform decision making in critical
applications like medical diagnosis and financial analysis have raised concerns
regarding their reliability and trustworthiness. As automated systems become
more mainstream, it is important their decisions be transparent, reliable and
understandable by humans for better trust and confidence. To this effect,
concept-based models such as Concept Bottleneck Models (CBMs) and
Self-Explaining Neural Networks (SENN) have been proposed which constrain the
latent space of a model to represent high level concepts easily understood by
domain experts in the field. Although concept-based models promise a good
approach to both increasing explainability and reliability, it is yet to be
shown if they demonstrate robustness and output consistent concepts under
systematic perturbations to their inputs. To better understand performance of
concept-based models on curated malicious samples, in this paper, we aim to
study their robustness to adversarial perturbations, which are also known as
the imperceptible changes to the input data that are crafted by an attacker to
fool a well-learned concept-based model. Specifically, we first propose and
analyze different malicious attacks to evaluate the security vulnerability of
concept based models. Subsequently, we propose a potential general adversarial
training-based defense mechanism to increase robustness of these systems to the
proposed malicious attacks. Extensive experiments on one synthetic and two
real-world datasets demonstrate the effectiveness of the proposed attacks and
the defense approach.

本研究旨在研究基于概念模型的鲁棒性和输出一致性的表现，提出并分析了不同的恶意攻击方式，并提出了基于对抗训练的防御机制，探讨了它们对模型在系统性扰动下的鲁棒性，发现所提出的防御方法可以提高概念模型的鲁棒性。