It is well established that neural networks with deep architectures perform better than shallow networks for many tasks in machine learning. In statistical physics, while there has been recent interest in representing physical data with generative modelling, the focus has been on shallow neural networks. A natural question to ask is whether deep neural networks hold any advantage over shallow networks in representing such data. We investigate this question by using unsupervised, generative graphical models to learn the probability distribution of a two-dimensional Ising system. Deep Boltzmann machines, deep belief networks, and deep restricted Boltzmann networks are trained on thermal spin configurations from this system, and compared to the shallow architecture of the restricted Boltzmann machine. We benchmark the models, focussing on the accuracy of generating energetic observables near the phase transition, where these quantities are most difficult to approximate. Interestingly, after training the generative networks, we observe that the accuracy essentially depends only on the number of neurons in the first hidden layer of the network, and not on other model details such as network depth or model type. This is evidence that shallow networks are more efficient than deep networks at representing physical probability distributions associated with Ising systems near criticality.

使用深度玻尔兹曼机、深度置信网络和深度限制玻尔兹曼网络对二维 Ising 系统进行非监督生成建模，比较与浅层架构的有限玻尔兹曼机的效果，并发现只有第一隐藏层的神经元数量对于生成能量观测量的准确度有影响，而架构的深度和模型类型对于准确度的影响很小，这证明在表示与 Ising 系统的临界概率分布相关的物理概率分布时，浅层网络比深度神经网络更有效率。

近临临界状态下的 Ising 模型深度学习