Recovering dynamical equations from observed noisy data is the central
challenge of system identification. We develop a statistical mechanical
approach to analyze sparse equation discovery algorithms, which typically
balance data fit and parsimony through a trial-and-error selection of
hyperparameters. In this framework, statistical mechanics offers tools to
analyze the interplay between complexity and fitness, in analogy to that done
between entropy and energy. To establish this analogy, we define the
optimization procedure as a two-level Bayesian inference problem that separates
variable selection from coefficient values and enables the computation of the
posterior parameter distribution in closed form. A key advantage of employing
statistical mechanical concepts, such as free energy and the partition
function, is in the quantification of uncertainty, especially in in the
low-data limit; frequently encountered in real-world applications. As the data
volume increases, our approach mirrors the thermodynamic limit, leading to
distinct sparsity- and noise-induced phase transitions that delineate correct
from incorrect identification. This perspective of sparse equation discovery,
is versatile and can be adapted to various other equation discovery algorithms.

从观察到的噪声数据中恢复动力学方程是系统辨识的核心挑战，我们开发了一种统计力学方法来分析稀疏方程发现算法，它通常通过试错选择超参数来平衡数据拟合和简洁性。在这种框架下，统计力学提供了工具来分析复杂性和适应性之间的相互作用，类似于熵和能量之间的分析。通过将优化过程定义为二级贝叶斯推断问题，将变量选择与系数值分离，并能够通过闭合形式计算后验参数分布，从而建立这种类比。运用统计力学的概念，如自由能和配分函数，在低数据限制下尤其能够量化不确定性，这在实际应用中经常遇到。随着数据量的增加，我们的方法类似于热力学极限，导致明确区分正确和错误辨识的不同疏松度和噪声诱导的相变。这种对稀疏方程发现的观点是多功能的，可以适应各种其他方程发现算法。