Explicit, momentum-based dynamics that optimize functions defined on Lie
groups can be constructed via variational optimization and momentum
trivialization. Structure preserving time discretizations can then turn this
dynamics into optimization algorithms. This article investigates two types of
discretization, Lie Heavy-Ball, which is a known splitting scheme, and Lie
NAG-SC, which is newly proposed. Their convergence rates are explicitly
quantified under $L$-smoothness and local strong convexity assumptions. Lie
NAG-SC provides acceleration over the momentumless case, i.e. Riemannian
gradient descent, but Lie Heavy-Ball does not. When compared to existing
accelerated optimizers for general manifolds, both Lie Heavy-Ball and Lie
NAG-SC are computationally cheaper and easier to implement, thanks to their
utilization of group structure. Only gradient oracle and exponential map are
required, but not logarithm map or parallel transport which are computational
costly.

通过变分优化和动量微化，可以构建优化 Lie 群上定义的函数的显式的基于动量的动力学方法。本文研究了两种离散化方法：Lie Heavy-Ball 和 Lie NAG-SC，分别提供了 L 平滑性和局部强凸性的显式收敛速度。与现有的一般流形加速优化器相比，Lie Heavy-Ball 和 Lie NAG-SC 都计算成本更低、更易实现，因为它们利用了群结构。只需要梯度预言子和指数映射，而不需要计算昂贵的对数映射或平行运输。

李群动量优化器的定量收敛性

Quantitative Convergences of Lie Group Momentum Optimizers

Conservation laws are well-established in the context of Euclidean gradient
flow dynamics, notably for linear or ReLU neural network training. Yet, their
existence and principles for non-Euclidean geometries and momentum-based
dynamics remain largely unknown. In this paper, we characterize "all"
conservation laws in this general setting. In stark contrast to the case of
gradient flows, we prove that the conservation laws for momentum-based dynamics
exhibit temporal dependence. Additionally, we often observe a "conservation
loss" when transitioning from gradient flow to momentum dynamics. Specifically,
for linear networks, our framework allows us to identify all momentum
conservation laws, which are less numerous than in the gradient flow case
except in sufficiently over-parameterized regimes. With ReLU networks, no
conservation law remains. This phenomenon also manifests in non-Euclidean
metrics, used e.g. for Nonnegative Matrix Factorization (NMF): all conservation
laws can be determined in the gradient flow context, yet none persists in the
momentum case.

本文研究了非欧几里德几何和基于动量的动力学中的守恒定律及其存在与原则，发现与梯度流情况不同，动量动力学中的守恒定律具有时间依赖性，转换为动量动力学时常常观察到守恒损失现象，线性网络中所有的动量守恒定律（数量小于梯度流情况）可以被确定，而 ReLU 网络中不存在守恒定律，这种现象也存在于用于非负矩阵分解（NMF）的非欧几里德度量中。

保持势能：欧几里得梯度流之外的守恒定律

Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows

Explicit, momentum-based dynamics for optimizing functions defined on Lie
groups was recently constructed, based on techniques such as variational
optimization and left trivialization. We appropriately add tractable noise to
the optimization dynamics to turn it into a sampling dynamics, leveraging the
advantageous feature that the momentum variable is Euclidean despite that the
potential function lives on a manifold. We then propose a Lie-group MCMC
sampler, by delicately discretizing the resulting kinetic-Langevin-type
sampling dynamics. The Lie group structure is exactly preserved by this
discretization. Exponential convergence with explicit convergence rate for both
the continuous dynamics and the discrete sampler are then proved under W2
distance. Only compactness of the Lie group and geodesically L-smoothness of
the potential function are needed. To the best of our knowledge, this is the
first convergence result for kinetic Langevin on curved spaces, and also the
first quantitative result that requires no convexity or, at least not
explicitly, any common relaxation such as isoperimetry.

基于动量的动力学方法的优化与采样机制的研究，其中涉及 Lie 群、变分优化、动量动力学与采样动力学，以及基于动量的动力学方法在曲面上的收敛性研究。