Boltzmann machines (BMs) are appealing candidates for powerful priors in
variational autoencoders (VAEs), as they are capable of capturing nontrivial
and multi-modal distributions over discrete variables. However,
non-differentiability of the discrete units prohibits using the
reparameterization trick, essential for low-noise back propagation. The Gumbel
trick resolves this problem in a consistent way by relaxing the variables and
distributions, but it is incompatible with BM priors. Here, we propose the
GumBolt, a model that extends the Gumbel trick to BM priors in VAEs. GumBolt is
significantly simpler than the recently proposed methods with BM prior and
outperforms them by a considerable margin. It achieves state-of-the-art
performance on permutation invariant MNIST and OMNIGLOT datasets in the scope
of models with only discrete latent variables. Moreover, the performance can be
further improved by allowing multi-sampled (importance-weighted) estimation of
log-likelihood in training, which was not possible with previous models.

该论文提出了一种名为 GumBolt 的模型，它扩展了 Gumbel 技巧来处理 VAEs 中的 Boltzmann 机先验，并在离散潜变量范围内的 MNIST 和 OMNIGLOT 数据集上实现了最先进的性能和重要性加权估计的日志似然训练等方面进一步提高了性能。

GumBolt：将 Gumbel 技巧扩展到 Boltzmann 先验

GumBolt: Extending Gumbel trick to Boltzmann priors

The Gumbel trick is a method to sample from a discrete probability
distribution, or to estimate its normalizing partition function. The method
relies on repeatedly applying a random perturbation to the distribution in a
particular way, each time solving for the most likely configuration. We derive
an entire family of related methods, of which the Gumbel trick is one member,
and show that the new methods have superior properties in several settings with
minimal additional computational cost. In particular, for the Gumbel trick to
yield computational benefits for discrete graphical models, Gumbel
perturbations on all configurations are typically replaced with so-called
low-rank perturbations. We show how a subfamily of our new methods adapts to
this setting, proving new upper and lower bounds on the log partition function
and deriving a family of sequential samplers for the Gibbs distribution.
Finally, we balance the discussion by showing how the simpler analytical form
of the Gumbel trick enables additional theoretical results.

该论文提出了一种基于 Gumbel trick 的采样方法，能够在最小的计算成本下，对离散概率分布进行采样和计算归一化分区函数，并在图模型方面得到了优越的性能。