Thanks to their simple architecture, Restricted Boltzmann Machines (RBMs) are
powerful tools for modeling complex systems and extracting interpretable
insights from data. However, training RBMs, as other energy-based models, on
highly structured data poses a major challenge, as effective training relies on
mixing the Markov chain Monte Carlo simulations used to estimate the gradient.
This process is often hindered by multiple second-order phase transitions and
the associated critical slowdown. In this paper, we present an innovative
method in which the principal directions of the dataset are integrated into a
low-rank RBM through a convex optimization procedure. This approach enables
efficient sampling of the equilibrium measure via a static Monte Carlo process.
By starting the standard training process with a model that already accurately
represents the main modes of the data, we bypass the initial phase transitions.
Our results show that this strategy successfully trains RBMs to capture the
full diversity of data in datasets where previous methods fail. Furthermore, we
use the training trajectories to propose a new sampling method, {\em parallel
trajectory tempering}, which allows us to sample the equilibrium measure of the
trained model much faster than previous optimized MCMC approaches and a better
estimation of the log-likelihood. We illustrate the success of the training
method on several highly structured datasets.

通过凸优化过程将数据集的主要方向整合到低秩 RBM 中，从而通过静态蒙特卡罗过程实现平衡分布的高效采样，成功训练 RBM 来捕捉之前方法失败的高度结构化数据集中的全部多样性，并提出了一种新的采样方法 - 平行轨迹调整，使得能够比之前的 MCMC 方法更快地采样训练模型的平衡分布并更好地估计对数似然。