We study the task of efficiently sampling from a Gibbs distribution $d \pi^*
= e^{-h} d {vol}_g$ over a Riemannian manifold $M$ via (geometric) Langevin
MCMC; this algorithm involves computing exponential maps in random Gaussian
directions and is efficiently implementable in practice. The key to our
analysis of Langevin MCMC is a bound on the discretization error of the
geometric Euler-Murayama scheme, assuming $\nabla h$ is Lipschitz and $M$ has
bounded sectional curvature. Our error bound matches the error of Euclidean
Euler-Murayama in terms of its stepsize dependence. Combined with a contraction
guarantee for the geometric Langevin Diffusion under Kendall-Cranston coupling,
we prove that the Langevin MCMC iterates lie within $\epsilon$-Wasserstein
distance of $\pi^*$ after $\tilde{O}(\epsilon^{-2})$ steps, which matches the
iteration complexity for Euclidean Langevin MCMC. Our results apply in general
settings where $h$ can be nonconvex and $M$ can have negative Ricci curvature.
Under additional assumptions that the Riemannian curvature tensor has bounded
derivatives, and that $\pi^*$ satisfies a $CD(\cdot,\infty)$ condition, we
analyze the stochastic gradient version of Langevin MCMC, and bound its
iteration complexity by $\tilde{O}(\epsilon^{-2})$ as well.

通过几何 Langevin MCMC 从一个 Riemann 流形 M 上的 Gibbs 分布 dπ* 进行高效采样的任务，我们提出了一种在实践中可实现的算法，该算法涉及在随机高斯方向上计算指数映射。通过对几何 Euler-Murayama 方案的离散化误差进行界定，假设▽h 是 Lipschitz 的且 M 具有有界的切向曲率，我们的误差界限与欧几里得 Euler-Murayama 的误差相匹配，结合 Kendall-Cranston 耦合下的几何 Langevin 扩散的收缩保证，我们证明 Langevin MCMC 迭代在经过～O (ε^-2) 次步骤后，与 π* 之间的 Wasserstein 距离小于 ε，这与欧几里得 Langevin MCMC 的迭代复杂性相匹配。我们的结果适用于具有非凸 h 和具有负 Ricci 曲率的一般设置。在额外的假设下，即 Riemann 曲率张量具有有界导数且 π* 满足 CD (・,∞) 条件，我们分析了 Langevin MCMC 的随机梯度版本，并将其迭代复杂性限制在～O (ε^-2) 次。

通过 Langevin MCMC 在 Riemann 流形上进行高效采样

Efficient Sampling on Riemannian Manifolds via Langevin MCMC

We show that the sum of the implicit generator log-density $\log p_g$ of a
GAN with the logit score of the discriminator defines an energy function which
yields the true data density when the generator is imperfect but the
discriminator is optimal, thus making it possible to improve on the typical
generator (with implicit density $p_g$). To make that practical, we show that
sampling from this modified density can be achieved by sampling in latent space
according to an energy-based model induced by the sum of the latent prior
log-density and the discriminator output score. This can be achieved by running
a Langevin MCMC in latent space and then applying the generator function, which
we call Discriminator Driven Latent Sampling~(DDLS). We show that DDLS is
highly efficient compared to previous methods which work in the
high-dimensional pixel space and can be applied to improve on previously
trained GANs of many types. We evaluate DDLS on both synthetic and real-world
datasets qualitatively and quantitatively. On CIFAR-10, DDLS substantially
improves the Inception Score of an off-the-shelf pre-trained
SN-GAN~\citep{sngan} from $8.22$ to $9.09$ which is even comparable to the
class-conditional BigGAN~\citep{biggan} model. This achieves a new
state-of-the-art in unconditional image synthesis setting without introducing
extra parameters or additional training.

本文提出了一种 Discriminator Driven Latent Sampling（DDLS）方法，使用 GAN 的隐含生成器 log 密度和鉴别器的 logit 分数定义一个能量函数，在生成器达到最优前，该函数表示最优真实数据密度。该方法在潜空间内运行 Langevin MCMC，并应用在生成器函数上，从而提高对多种类型的 GAN 进行训练的效率，并无需引入额外的参数或训练。在评估上达到了无条件图像合成的新的最新技术。

Your GAN 是悄悄变成了能量基模型，你应该使用判别器驱动的隐变量采样

Your GAN is Secretly an Energy-based Model and You Should use  Discriminator Driven Latent Sampling

We study the problem of sampling from a distribution $p^*(x) \propto
\exp\left(-U(x)\right)$, where the function $U$ is $L$-smooth everywhere and
$m$-strongly convex outside a ball of radius $R$, but potentially nonconvex
inside this ball. We study both overdamped and underdamped Langevin MCMC and
establish upper bounds on the number of steps required to obtain a sample from
a distribution that is within $\epsilon$ of $p^*$ in $1$-Wasserstein distance.
For the first-order method (overdamped Langevin MCMC), the iteration complexity
is $\tilde{\mathcal{O}}\left(e^{cLR^2}d/\epsilon^2\right)$, where $d$ is the
dimension of the underlying space. For the second-order method (underdamped
Langevin MCMC), the iteration complexity is
$\tilde{\mathcal{O}}\left(e^{cLR^2}\sqrt{d}/\epsilon\right)$ for an explicit
positive constant $c$. Surprisingly, the iteration complexity for both these
algorithms is only polynomial in the dimension $d$ and the target accuracy
$\epsilon$. It is exponential, however, in the problem parameter $LR^2$, which
is a measure of non-log-concavity of the target distribution.

本研究研究了在采样中采用了过阻尼和欠阻尼 Langevin MCMC，证明了算法的迭代复杂度在维度和目标准确度方面均是多项式级别的，但在问题参数 LR ^ 2 中是指数级别的，从而可以更好地进行非凸优化。