Interpretability methods for deep neural networks mainly focus on the
sensitivity of the class score with respect to the original or perturbed input,
usually measured using actual or modified gradients. Some methods also use a
model-agnostic approach to understanding the rationale behind every prediction.
In this paper, we argue and demonstrate that local geometry of the model
parameter space relative to the input can also be beneficial for improved
post-hoc explanations. To achieve this goal, we introduce an interpretability
method called "geometrically-guided integrated gradients" that builds on top of
the gradient calculation along a linear path as traditionally used in
integrated gradient methods. However, instead of integrating gradient
information, our method explores the model's dynamic behavior from multiple
scaled versions of the input and captures the best possible attribution for
each input. We demonstrate through extensive experiments that the proposed
approach outperforms vanilla and integrated gradients in subjective and
quantitative assessment. We also propose a "model perturbation" sanity check to
complement the traditionally used "model randomization" test.

本文提出了一种基于深度神经网络模型参数空间局部几何性质的解释性方法 —— 几何引导的积分梯度法，在主观和量化评价方面表现优于传统的积分梯度法和总体梯度法，并提出了 “模型扰动” 的健全性检查，以补充传统的 “模型随机化” 测试。

几何引导式的整合梯度

Geometrically Guided Integrated Gradients

In this paper we consider a problem of searching a space of predictive models
for a given training data set. We propose an iterative procedure for deriving a
sequence of improving models and a corresponding sequence of sets of non-linear
features on the original input space. After a finite number of iterations N,
the non-linear features become 2^N -degree polynomials on the original space.
We show that in a limit of an infinite number of iterations derived non-linear
features must form an associative algebra: a product of two features is equal
to a linear combination of features from the same feature space for any given
input point. Because each iteration consists of solving a series of convex
problems that contain all previous solutions, the likelihood of the models in
the sequence is increasing with each iteration while the dimension of the model
parameter space is set to a limited controlled value.

本文研究的问题是在给定的训练数据集中寻找预测模型。我们提出了一种迭代过程，用于得到一系列不断改进的模型和对原始输入空间的一系列非线性特征。在有限的 N 次迭代后，非线性特征变成原始空间上 2^N - 次多项式。我们展示了在无限次迭代的极限情况下，导出的非线性特征必须形成一个联想代数：两个特征的乘积等于来自同一特征空间的特征的线性组合。由于每次迭代都包含解决一系列包含所有先前解的凸问题，因此随着每次迭代，模型的似然性逐渐提高，而模型参数空间的维数设置为一个有限的可控制值。