Research in both machine learning and psychology suggests that salient
examples can help humans to interpret learning models. To this end, we take a
novel look at black box interpretation of test predictions in terms of training
examples. Our goal is to ask `which training examples are most responsible for
a given set of predictions'? To answer this question, we make use of Fisher
kernels as the defining feature embedding of each data point, combined with
Sequential Bayesian Quadrature (SBQ) for efficient selection of examples. In
contrast to prior work, our method is able to seamlessly handle any sized
subset of test predictions in a principled way. We theoretically analyze our
approach, providing novel convergence bounds for SBQ over discrete candidate
atoms. Our approach recovers the application of influence functions for
interpretability as a special case yielding novel insights from this
connection. We also present applications of the proposed approach to three use
cases: cleaning training data, fixing mislabeled examples and data
summarization.

通过使用 Fisher kernels 和 SBQ 算法，我们提出了一种新的方法来解释黑盒机器学习模型，其可以无限制地处理任何大小的测试数据，包括清理训练数据，修复错误标记的示例和数据总结。

利用 Fisher 核解释黑盒预测

Interpreting Black Box Predictions using Fisher Kernels

Batch Normalization (BN) is essential to effectively train state-of-the-art
deep Convolutional Neural Networks (CNN). It normalizes inputs to the layers
during training using the statistics of each mini-batch. In this work, we study
BN from the viewpoint of Fisher kernels. We show that assuming samples within a
mini-batch are from the same probability density function, then BN is identical
to the Fisher vector of a Gaussian distribution. That means BN can be explained
in terms of kernels that naturally emerge from the probability density function
of the underlying data distribution. However, given the rectifying
non-linearities employed in CNN architectures, distribution of inputs to the
layers show heavy tail and asymmetric characteristics. Therefore, we propose
approximating underlying data distribution not with one, but a mixture of
Gaussian densities. Deriving Fisher vector for a Gaussian Mixture Model (GMM),
reveals that BN can be improved by independently normalizing with respect to
the statistics of disentangled sub-populations. We refer to our proposed soft
piecewise version of BN as Mixture Normalization (MN). Through extensive set of
experiments on CIFAR-10 and CIFAR-100, we show that MN not only effectively
accelerates training image classification and Generative Adversarial networks,
but also reaches higher quality models.

该研究使用 Fisher 核的角度探索 Batch Normalization 的优化，提出了一种新的 Mixture Normalization 方法，通过 CIFAR-10 和 CIFAR-100 实验验证能够有效加速深度神经网络模型的训练