Shapley values are widely used to explain black-box models, but they are
costly to calculate because they require many model evaluations. We introduce
FastSHAP, a method for estimating Shapley values in a single forward pass using
a learned explainer model. FastSHAP amortizes the cost of explaining many
inputs via a learning approach inspired by the Shapley value's weighted least
squares characterization, and it can be trained using standard stochastic
gradient optimization. We compare FastSHAP to existing estimation approaches,
revealing that it generates high-quality explanations with orders of magnitude
speedup.

FastSHAP 是一种使用学习解释器模型在一次正向传递中估计 Shapley 值的方法，通过启发 Shapley 值加权最小二乘估计的学习方法来分摊解释许多输入的成本，并且可以使用标准的随机梯度优化进行训练，与现有的估计方法进行比较，显示出具有高质量解释的数量级加速。

FastSHAP: 实时 Shapley 值估计

FastSHAP: Real-Time Shapley Value Estimation

We introduce a gradient-based learning method to automatically adapt Markov
chain Monte Carlo (MCMC) proposal distributions to intractable targets. We
define a maximum entropy regularised objective function, referred to as
generalised speed measure, which can be robustly optimised over the parameters
of the proposal distribution by applying stochastic gradient optimisation. An
advantage of our method compared to traditional adaptive MCMC methods is that
the adaptation occurs even when candidate state values are rejected. This is a
highly desirable property of any adaptation strategy because the adaptation
starts in early iterations even if the initial proposal distribution is far
from optimum. We apply the framework for learning multivariate random walk
Metropolis and Metropolis-adjusted Langevin proposals with full covariance
matrices, and provide empirical evidence that our method can outperform other
MCMC algorithms, including Hamiltonian Monte Carlo schemes.

本研究提出使用基于梯度的学习方法来自适应马尔可夫链蒙特卡罗（MCMC）提议分布，应用随机梯度优化能通过定义的最大熵正则化目标函数来优化提议分布的参数，并证明相比传统自适应 MCMC 方法，该方法带有更高的性能；并应用于多元随机步长 Metropolis 和 Metropolis-adjusted Langevin 提议与完整协方差矩阵，并证实该方法在 MCMC 算法中表现优异，包括哈密顿蒙特卡罗方案。

基于梯度的自适应马尔科夫链蒙特卡罗

Gradient-based Adaptive Markov Chain Monte Carlo

Modern deep neural network training is typically based on mini-batch
stochastic gradient optimization. While the use of large mini-batches increases
the available computational parallelism, small batch training has been shown to
provide improved generalization performance and allows a significantly smaller
memory footprint, which might also be exploited to improve machine throughput.
In this paper, we review common assumptions on learning rate scaling and
training duration, as a basis for an experimental comparison of test
performance for different mini-batch sizes. We adopt a learning rate that
corresponds to a constant average weight update per gradient calculation (i.e.,
per unit cost of computation), and point out that this results in a variance of
the weight updates that increases linearly with the mini-batch size $m$.
The collected experimental results for the CIFAR-10, CIFAR-100 and ImageNet
datasets show that increasing the mini-batch size progressively reduces the
range of learning rates that provide stable convergence and acceptable test
performance. On the other hand, small mini-batch sizes provide more up-to-date
gradient calculations, which yields more stable and reliable training. The best
performance has been consistently obtained for mini-batch sizes between $m = 2$
and $m = 32$, which contrasts with recent work advocating the use of mini-batch
sizes in the thousands.

本文研究了基于小批量样本随机梯度优化的现代深度神经网络训练中，不同批量大小对测试性能和泛化性能的影响，并指出使用小批量训练可以提供更稳定和可靠的结果。