Stochastic Gradient Langevin Dynamics (SGLD) is a sampling scheme for
Bayesian modeling adapted to large datasets and models. SGLD relies on the
injection of Gaussian Noise at each step of a Stochastic Gradient Descent (SGD)
update. In this scheme, every component in the noise vector is independent and
has the same scale, whereas the parameters we seek to estimate exhibit strong
variations in scale and significant correlation structures, leading to poor
convergence and mixing times. We compare different preconditioning approaches
to the normalization of the noise vector and benchmark these approaches on the
following criteria: 1) mixing times of the multivariate parameter vector, 2)
regularizing effect on small dataset where it is easy to overfit, 3) covariate
shift detection and 4) resistance to adversarial examples.

本文比较了不同的预处理方法以规范噪声向量，从混合时间、正则化效果、协变量漂移检测和对抗样本抗干扰性等方面对其进行了评估。

实际中的可扩展自然梯度 Langevin 动力学

Scalable Natural Gradient Langevin Dynamics in Practice

Gibbs sampling is a Markov Chain Monte Carlo sampling technique that
iteratively samples variables from their conditional distributions. There are
two common scan orders for the variables: random scan and systematic scan. Due
to the benefits of locality in hardware, systematic scan is commonly used, even
though most statistical guarantees are only for random scan. While it has been
conjectured that the mixing times of random scan and systematic scan do not
differ by more than a logarithmic factor, we show by counterexample that this
is not the case, and we prove that that the mixing times do not differ by more
than a polynomial factor under mild conditions. To prove these relative bounds,
we introduce a method of augmenting the state space to study systematic scan
using conductance.

使用 conductance 方法增加状态空间进行研究，证明从系统扫描中得到的混合时间与从随机扫描中得到的混合时间最多相差多项式阶。

Gibbs 抽样中的扫描顺序：在扫描顺序有影响的模型中以及其影响程度的界限

Scan Order in Gibbs Sampling: Models in Which it Matters and Bounds on  How Much

The main results in this paper are about the full coalescence time
$\mathsf{C}$ of a system of coalescing random walks over a finite graph $G$.
Letting $\mathsf{m}(G)$ denote the mean meeting time of two such walkers, we
give sufficient conditions under which $\mathbf{E}[\mathsf{C}]\approx
2\mathsf{m}(G)$ and $\mathsf{C}/\mathsf{m}(G)$ has approximately the same law
as in the "mean field" setting of a large complete graph. One of our theorems
is that mean field behavior occurs over all vertex-transitive graphs whose
mixing times are much smaller than $\mathsf{m}(G)$; this nearly solves an open
problem of Aldous and Fill and also generalizes results of Cox for discrete
tori in $d\geq2$ dimensions. Other results apply to nonreversible walks and
also generalize previous theorems of Durrett and Cooper et al. Slight
extensions of these results apply to voter model consensus times, which are
related to coalescing random walks via duality. Our main proof ideas are a
strengthening of the usual approximation of hitting times by exponential random
variables, which give results for nonstationary initial states; and a new
general set of conditions under which we can prove that the hitting time of a
union of sets behaves like a minimum of independent exponentials. In
particular, this will show that the first meeting time among $k$ random walkers
has mean $\approx\mathsf{m}(G)/\bigl({\matrix{k 2}}\bigr)$.

本文研究了有限图上合并随机行走的完全合并时间 C，提出了 mean meeting time 和 mixing times 的充分条件，得出了类似于大型完整图的定律。证实均场行为可在混合时间远小于 mean meeting time 的所有顶点可转移图中出现，并且扩展到非翻转游走和选民模型共识时间。