Restricted Boltzmann Machines (RBMs) and models derived from them have been
successfully used as basic building blocks in deep artificial neural networks
for automatic features extraction, unsupervised weights initialization, but
also as density estimators. Thus, their generative and discriminative
capabilities, but also their computational time are instrumental to a wide
range of applications. Our main contribution is to look at RBMs from a
topological perspective, bringing insights from network science. Firstly, here
we show that RBMs and Gaussian RBMs (GRBMs) are bipartite graphs which
naturally have a small-world topology. Secondly, we demonstrate both on
synthetic and real-world datasets that by constraining RBMs and GRBMs to a
scale-free topology (while still considering local neighborhoods and data
distribution), we reduce the number of weights that need to be computed by a
few orders of magnitude, at virtually no loss in generative performance.
Thirdly, we show that, for a fixed number of weights, our proposed sparse
models (which by design have a higher number of hidden neurons) achieve better
generative capabilities than standard fully connected RBMs and GRBMs (which by
design have a smaller number of hidden neurons), at no additional computational
costs.

本文从网络科学的角度，提出了一种基于小世界和无标度网络拓扑结构的约束 Restricted Boltzmann Machines 模型，它能够大大减少权重数量，提高生成能力，而不增加计算成本。