We revisit the challenging problem of training Gaussian-Bernoulli restricted
Boltzmann machines (GRBMs), introducing two innovations. We propose a novel
Gibbs-Langevin sampling algorithm that outperforms existing methods like Gibbs
sampling. We propose a modified contrastive divergence (CD) algorithm so that
one can generate images with GRBMs starting from noise. This enables direct
comparison of GRBMs with deep generative models, improving evaluation protocols
in the RBM literature. Moreover, we show that modified CD and gradient clipping
are enough to robustly train GRBMs with large learning rates, thus removing the
necessity of various tricks in the literature. Experiments on Gaussian
Mixtures, MNIST, FashionMNIST, and CelebA show GRBMs can generate good samples,
despite their single-hidden-layer architecture. Our code is released at:
https://github.com/lrjconan/GRBM.

我们提出了一种新颖的 Gibbs-Langewin 采样算法和改进的对比散度算法，可用于训练 Gaussian-Bernoulli restricted Boltzmann machines (GRBMs)。实验结果表明，我们的方法可以有效地训练 GRBMs，并且与深层生成模型进行了直接比较。