We interpret the predictions of any black-box structured input-structured
output model around a specific input-output pair. Our method returns an
"explanation" consisting of groups of input-output tokens that are causally
related. These dependencies are inferred by querying the black-box model with
perturbed inputs, generating a graph over tokens from the responses, and
solving a partitioning problem to select the most relevant components. We focus
the general approach on sequence-to-sequence problems, adopting a variational
autoencoder to yield meaningful input perturbations. We test our method across
several NLP sequence generation tasks.

采用黑盒结构化输入输出模型进行预测并通过干扰输入生成相关性图和解决分割问题来解释黑盒模型预测，着重于序列生成问题并采用变分自编码器来产生有意义的干扰，我们在多个 NLP 序列 - 生成任务中测试了该方法。

解释黑盒序列到序列模型预测的因果框架

A causal framework for explaining the predictions of black-box  sequence-to-sequence models

How can we explain the predictions of a black-box model? In this paper, we
use influence functions -- a classic technique from robust statistics -- to
trace a model's prediction through the learning algorithm and back to its
training data, thereby identifying training points most responsible for a given
prediction. To scale up influence functions to modern machine learning
settings, we develop a simple, efficient implementation that requires only
oracle access to gradients and Hessian-vector products. We show that even on
non-convex and non-differentiable models where the theory breaks down,
approximations to influence functions can still provide valuable information.
On linear models and convolutional neural networks, we demonstrate that
influence functions are useful for multiple purposes: understanding model
behavior, debugging models, detecting dataset errors, and even creating
visually-indistinguishable training-set attacks.

本文提出使用影响函数技术来追踪黑盒模型预测的训练数据，并且证明了即使在非凸和不可导的模型中，影响函数的近似也可以提供有价值的信息。在线性模型和卷积神经网络中，通过使用影响函数，探究了模型行为、调试模型、检测数据集错误以及创建视觉上难以区分的训练集攻击等多个方面得到了实际的解决方案。

通过影响函数理解黑盒预测

Understanding Black-box Predictions via Influence Functions

We extend the work of Lehre and Witt (GECCO 2010) on the unbiased black-box
model by considering higher arity variation operators. In particular, we show
that already for binary operators the black-box complexity of \leadingones
drops from $\Theta(n^2)$ for unary operators to $O(n \log n)$. For \onemax, the
$\Omega(n \log n)$ unary black-box complexity drops to O(n) in the binary case.
For $k$-ary operators, $k \leq n$, the \onemax-complexity further decreases to
$O(n/\log k)$.

本文研究了高级别变异算子在无偏黑箱模型上的应用，特别地，通过研究二进制算子对 leading-ones 和 onemax 的影响，发现 unary 黑箱模型的时间复杂度从 Ω（nlogn）降至 O（nlogn），而 binary 模型的时间复杂度下降到了 O（n）。同时针对 k-ary 操作，本文还发现了时间复杂度能进一步减小至 O（n /log k）.