Restricted Boltzmann Machines (RBMs) and models derived from them have been
successfully used as basic building blocks in deep artificial neural networks
for automatic features extraction, unsupervised weights initialization, but
also as density estimators. Thus, their generative and discriminative
capabilities, but also their computational time are instrumental to a wide
range of applications. Our main contribution is to look at RBMs from a
topological perspective, bringing insights from network science. Firstly, here
we show that RBMs and Gaussian RBMs (GRBMs) are bipartite graphs which
naturally have a small-world topology. Secondly, we demonstrate both on
synthetic and real-world datasets that by constraining RBMs and GRBMs to a
scale-free topology (while still considering local neighborhoods and data
distribution), we reduce the number of weights that need to be computed by a
few orders of magnitude, at virtually no loss in generative performance.
Thirdly, we show that, for a fixed number of weights, our proposed sparse
models (which by design have a higher number of hidden neurons) achieve better
generative capabilities than standard fully connected RBMs and GRBMs (which by
design have a smaller number of hidden neurons), at no additional computational
costs.

本文从网络科学的角度，提出了一种基于小世界和无标度网络拓扑结构的约束 Restricted Boltzmann Machines 模型，它能够大大减少权重数量，提高生成能力，而不增加计算成本。

限制玻尔兹曼机的拓扑洞察

A topological insight into restricted Boltzmann machines

The Saccharomyces cerevisiae protein-protein interaction map, as well as many
natural and man-made networks, shares the scale-free topology. The preferential
attachment model was suggested as a generic network evolution model that yields
this universal topology. However, it is not clear that the model assumptions
hold for the protein interaction network. Using a cross genome comparison we
show that (a) the older a protein, the better connected it is, and (b) The
number of interactions a protein gains during its evolution is proportional to
its connectivity. Therefore, preferential attachment governs the protein
network evolution. The evolutionary mechanism leading to such preference and
some implications are discussed.

通过跨基因组比较，我们表明优先附着模型主宰蛋白质网络的进化，并讨论了导致这种偏好的进化机制及其一些影响。