In the field of medical imaging, particularly in tasks related to early
disease detection and prognosis, understanding the reasoning behind AI model
predictions is imperative for assessing their reliability. Conventional
explanation methods encounter challenges in identifying decisive features in
medical image classifications, especially when discriminative features are
subtle or not immediately evident. To address this limitation, we propose an
agent model capable of generating counterfactual images that prompt different
decisions when plugged into a black box model. By employing this agent model,
we can uncover influential image patterns that impact the black model's final
predictions. Through our methodology, we efficiently identify features that
influence decisions of the deep black box. We validated our approach in the
rigorous domain of medical prognosis tasks, showcasing its efficacy and
potential to enhance the reliability of deep learning models in medical image
classification compared to existing interpretation methods. The code will be
publicly available at this https URL

通过生成对抗图像，我们提出了一种方法来揭示影响黑盒模型的决策的关键特征，以提高医学图像分类中深度学习模型的可靠性。

DiffExplainer: 通过反事实生成揭示黑匣子模型

DiffExplainer: Unveiling Black Box Models Via Counterfactual Generation

Neural networks are prone to learning shortcuts -- they often model simple
correlations, ignoring more complex ones that potentially generalize better.
Prior works on image classification show that instead of learning a connection
to object shape, deep classifiers tend to exploit spurious correlations with
low-level texture or the background for solving the classification task. In
this work, we take a step towards more robust and interpretable classifiers
that explicitly expose the task's causal structure. Building on current
advances in deep generative modeling, we propose to decompose the image
generation process into independent causal mechanisms that we train without
direct supervision. By exploiting appropriate inductive biases, these
mechanisms disentangle object shape, object texture, and background; hence,
they allow for generating counterfactual images. We demonstrate the ability of
our model to generate such images on MNIST and ImageNet. Further, we show that
the counterfactual images can improve out-of-distribution robustness with a
marginal drop in performance on the original classification task, despite being
synthetic. Lastly, our generative model can be trained efficiently on a single
GPU, exploiting common pre-trained models as inductive biases.

通过深度生成建模，我们将图像生成过程分解成独立的因果机制，从而产生反事实图像，提高了模型的新颖性和可解释性。在 MNIST 和 ImageNet 上的实验表明，这些反事实图像能够提高模型的抗干扰性，而且这个模型能在单 GPU 上高效训练。

反事实生成网络

Counterfactual Generative Networks

While Visual Question Answering (VQA) models continue to push the
state-of-the-art forward, they largely remain black-boxes - failing to provide
insight into how or why an answer is generated. In this ongoing work, we
propose addressing this shortcoming by learning to generate counterfactual
images for a VQA model - i.e. given a question-image pair, we wish to generate
a new image such that i) the VQA model outputs a different answer, ii) the new
image is minimally different from the original, and iii) the new image is
realistic. Our hope is that providing such counterfactual examples allows users
to investigate and understand the VQA model's internal mechanisms.

通过学习生成对抗性图片对视觉问答模型进行解释，从而使得人们可以理解其内部机制。