The algorithms used to train neural networks, like stochastic gradient
descent (SGD), have close parallels to natural processes that navigate a
high-dimensional parameter space -- for example protein folding or evolution.
Our study uses a Fokker-Planck approach, adapted from statistical physics, to
explore these parallels in a single, unified framework. We focus in particular
on the stationary state of the system in the long-time limit, which in
conventional SGD is out of equilibrium, exhibiting persistent currents in the
space of network parameters. As in its physical analogues, the current is
associated with an entropy production rate for any given training trajectory.
The stationary distribution of these rates obeys the integral and detailed
fluctuation theorems -- nonequilibrium generalizations of the second law of
thermodynamics. We validate these relations in two numerical examples, a
nonlinear regression network and MNIST digit classification. While the
fluctuation theorems are universal, there are other aspects of the stationary
state that are highly sensitive to the training details. Surprisingly, the
effective loss landscape and diffusion matrix that determine the shape of the
stationary distribution vary depending on the simple choice of minibatching
done with or without replacement. We can take advantage of this nonequilibrium
sensitivity to engineer an equilibrium stationary state for a particular
application: sampling from a posterior distribution of network weights in
Bayesian machine learning. We propose a new variation of stochastic gradient
Langevin dynamics (SGLD) that harnesses without replacement minibatching. In an
example system where the posterior is exactly known, this SGWORLD algorithm
outperforms SGLD, converging to the posterior orders of magnitude faster as a
function of the learning rate.

本研究探索了神经网络训练算法与自然过程如蛋白质折叠和进化之间的相似性，使用统计物理中 Fokker-Planck 方法将它们在一个统一的框架下探索，研究了在长时间极限下系统的稳态和出现的熵产生率，验证了涉及到这些数值的图谱存在扰动定理，提出了一种新的随机梯度 Langevin 动力学（SGLD）算法，可以应用于贝叶斯机器学习中从后验分布中获取网络权重。

机器学习的平衡状态内外

Machine learning in and out of equilibrium

Stochastic thermodynamics as reviewed here systematically provides a
framework for extending the notions of classical thermodynamics like work, heat
and entropy production to the level of individual trajectories of well-defined
non-equilibrium ensembles. It applies whenever a non-equilibrium process is
still coupled to one (or several) heat bath(s) of constant temperature.
Paradigmatic systems are single colloidal particles in time-dependent laser
traps, polymers in external flow, enzymes and molecular motors in single
molecule assays, small biochemical networks and thermoelectric devices
involving single electron transport. For such systems, a first-law like energy
balance can be identified along fluctuating trajectories. Various integral and
detailed fluctuation theorems, which are derived here in a unifying approach
from one master theorem, constrain the probability distributions for work, heat
and entropy production depending on the nature of the system and the choice of
non-equilibrium conditions. For non-equilibrium steady states, particularly
strong results hold like a generalized fluctuation-dissipation theorem
involving entropy production. Ramifications and applications of these concepts
include optimal driving between specified states in finite time, the role of
measurement-based feedback processes and the relation between dissipation and
irreversibility. Efficiency and, in particular, efficiency at maximum power,
can be discussed systematically beyond the linear response regime for two
classes of molecular machines, isothermal ones like molecular motors, and heat
engines like thermoelectric devices, using a common framework based on a cycle
decomposition of entropy production.

随机热力学是将经典热力学的概念（如功、热以及熵的产生）扩展到非平衡集合的个体轨迹的框架，主要应用于单胶体颗粒、聚合物、酶和分子电机、小型生化网络以及热电器件等系统中，各种波动定理限制着功、热和熵产生的概率分布，并可以通过一个主定理来规定。