Restricted Boltzmann Machines are generative models that consist of a layer
of hidden variables connected to another layer of visible units, and they are
used to model the distribution over visible variables. In order to gain a
higher representability power, many hidden units are commonly used, which, in
combination with a large number of visible units, leads to a high number of
trainable parameters. In this work we introduce the Structural Restricted
Boltzmann Machine model, which taking advantage of the structure of the data in
hand, constrains connections of hidden units to subsets of visible units in
order to reduce significantly the number of trainable parameters, without
compromising performance. As a possible area of application, we focus on image
modelling. Based on the nature of the images, the structure of the connections
is given in terms of spatial neighbourhoods over the pixels of the image that
constitute the visible variables of the model. We conduct extensive experiments
on various image domains. Image denoising is evaluated with corrupted images
from the MNIST dataset. The generative power of our models is compared to
vanilla RBMs, as well as their classification performance, which is assessed
with five different image domains. Results show that our proposed model has a
faster and more stable training, while also obtaining better results compared
to an RBM with no constrained connections between its visible and hidden units.

本文介绍的结构受限玻尔兹曼机（Structural Restricted Boltzmann Machine）模型针对图像建模的应用，通过限制隐层单元与可见层子集的连接来显著降低可训练参数数量，而不会影响性能。通过对多个图像域进行广泛实验，评估使用 MNIST 数据集的图像去噪结果和五个不同图像域的分类性能，结果表明相比于无限制连接的 RBM，该模型具有更快、更稳定的训练，并且可以获得更好的结果。

基于结构的受限玻尔兹曼机图像去噪与分类

Structural Restricted Boltzmann Machine for image denoising and  classification

We propose a novel approximate inference algorithm that approximates a target
distribution by amortising the dynamics of a user-selected MCMC sampler. The
idea is to initialise MCMC using samples from an approximation network, apply
the MCMC operator to improve these samples, and finally use the samples to
update the approximation network thereby improving its quality. This provides a
new generic framework for approximate inference, allowing us to deploy highly
complex, or implicitly defined approximation families with intractable
densities, including approximations produced by warping a source of randomness
through a deep neural network. Experiments consider image modelling with deep
generative models as a challenging test for the method. Deep models trained
using amortised MCMC are shown to generate realistic looking samples as well as
producing diverse imputations for images with regions of missing pixels.

提出了一种新的近似推断算法，通过按摩用户选择的 MCMC 采样器的动态来近似目标分布，并使用这些样本更新近似网络，从而提高其质量，进而允许高度复杂或隐含定义的近似族，包括通过深度神经网络扭曲随机性源而产生的近似。实验以使用深度生成模型的图像建模为测试。通过摊销 MCMC 训练的深度模型被证明可以生成逼真的图像样本，以及为具有缺失像素区域的图像产生多样的填充结果。

近似推理的摊还 MCMC

Approximate Inference with Amortised MCMC

We present an architecture which lets us train deep, directed generative
models with many layers of latent variables. We include deterministic paths
between all latent variables and the generated output, and provide a richer set
of connections between computations for inference and generation, which enables
more effective communication of information throughout the model during
training. To improve performance on natural images, we incorporate a
lightweight autoregressive model in the reconstruction distribution. These
techniques permit end-to-end training of models with 10+ layers of latent
variables. Experiments show that our approach achieves state-of-the-art
performance on standard image modelling benchmarks, can expose latent class
structure in the absence of label information, and can provide convincing
imputations of occluded regions in natural images.

本文介绍了一种基于深度生成模型的架构，该架构在训练过程中使用多层的潜在变量，并且在所有潜在变量和输出之间包括确定性路径，以及提供了一组更丰富的连接，这些连接在推理和生成计算之间实现了更有效的信息通信。我们在重构分布中引入了轻量级自回归模型以提高自然图像的性能。实验表明，我们的方法在标准图像建模基准测试中达到了最先进的性能，并且在没有标签信息的情况下可以暴露潜在的类结构，同时可以提供自然图像中被遮挡区域的令人信服的估计。