Mathematical equivalence between statistical mechanics and machine learning
theory has been known since the 20th century, and researches based on such
equivalence has provided novel methodology in both theoretical physics and
statistical learning theory. For example, algebraic approach in statistical
mechanics such as operator algebra enables us to analyze phase transition
phenomena mathematically. In this paper, for theoretical physicists who are
interested in artificial intelligence, we review and prospect algebraic
researches in machine learning theory. If a learning machine has hierarchical
structure or latent variables, then the random Hamiltonian cannot be expressed
by any quadratic perturbation because it has singularities. To study an
equilibrium state defined by such a singular random Hamiltonian, algebraic
approach is necessary to derive asymptotic form of the free energy and the
generalization error. We also introduce the most recent advance, in
fact,theoretical foundation for alignment of artificial intelligence is now
being constructed based on algebraic learning theory. This paper is devoted to
the memory of Professor Huzihiro Araki who is a pioneer founder of algebraic
research in both statistical mechanics and quantum field theory.

数理力学和机器学习理论之间的数学等价性是自 20 世纪以来已知的，基于这种等价性的研究为理论物理和统计学习理论提供了新的方法论。本文回顾和展望了机器学习理论中的代数研究，介绍了最近的进展，即正在基于代数学习理论构建人工智能的理论基础。

统计力学与机器学习理论之间等价框架的代数研究回顾与展望

Review and Prospect of Algebraic Research in Equivalent Framework  between Statistical Mechanics and Machine Learning Theory

Statistical learning theory and high dimensional statistics have had a
tremendous impact on Machine Learning theory and have impacted a variety of
domains including systems and control theory. Over the past few years we have
witnessed a variety of applications of such theoretical tools to help answer
questions such as: how many state-action pairs are needed to learn a static
control policy to a given accuracy? Recent results have shown that continuously
differentiable and stabilizing control policies can be well-approximated using
neural networks with hard guarantees on performance, yet often even the
simplest constrained control problems are not smooth. To address this void, in
this paper we study smooth approximations of linear Model Predictive Control
(MPC) policies, in which hard constraints are replaced by barrier functions,
a.k.a. barrier MPC. In particular, we show that barrier MPC inherits the
exponential stability properties of the original non-smooth MPC policy. Using a
careful analysis of the proposed barrier MPC, we show that its smoothness
constant can be carefully controlled, thereby paving the way for new sample
complexity results for approximating MPC policies from sampled state-action
pairs.

本论文研究采用阻碍函数作为硬约束的线性模型预测控制策略的平滑逼近，通过精细的分析展示其平滑常数可以被仔细控制，从而为采样状态 - 动作对逼近 MPC 策略的新样本复杂度结果铺平道路

应用于统计学习的平滑模型预测控制

Smooth Model Predictive Control with Applications to Statistical  Learning

This work connects language model adaptation with concepts of machine
learning theory. We consider a training setup with a large out-of-domain set
and a small in-domain set. We derive how the benefit of training a model on
either set depends on the size of the sets and the distance between their
underlying distributions. We analyze how out-of-domain pre-training before
in-domain fine-tuning achieves better generalization than either solution
independently. Finally, we present how adaptation techniques based on data
selection, such as importance sampling, intelligent data selection and
influence functions, can be presented in a common framework which highlights
their similarity and also their subtle differences.

探讨了语言模型适应与机器学习理论的关系，研究了大型领域外训练集和小型领域内训练集之间的训练方法的优劣，提出了领域外预训练加上领域内微调比单独应用更为通用，并提出了基于数据选择的适应技术的公共框架。