Sequential-in-time methods solve a sequence of training problems to fit
nonlinear parametrizations such as neural networks to approximate solution
trajectories of partial differential equations over time. This work shows that
sequential-in-time training methods can be understood broadly as either
optimize-then-discretize (OtD) or discretize-then-optimize (DtO) schemes, which
are well known concepts in numerical analysis. The unifying perspective leads
to novel stability and a posteriori error analysis results that provide
insights into theoretical and numerical aspects that are inherent to either OtD
or DtO schemes such as the tangent space collapse phenomenon, which is a form
of over-fitting. Additionally, the unified perspective facilitates establishing
connections between variants of sequential-in-time training methods, which is
demonstrated by identifying natural gradient descent methods on energy
functionals as OtD schemes applied to the corresponding gradient flows.

顺序时间方法用于训练非线性参数化模型（如神经网络）以近似求解偏微分方程随时间变化的解轨迹。本文指出顺序时间训练方法可以广义地理解为优化 - 离散化或离散化 - 优化的方案，并将其与数值分析中已知的概念联系起来。其统一视角提供了新的稳定性和后验误差分析结果，从而揭示了优化 - 离散化或离散化 - 优化方案固有的理论和数值方面的洞察，如切空间崩溃现象（一种过拟合形式）。此外，统一视角有助于建立顺序时间训练方法的不同变体之间的连接，例如将能量泛函上的自然梯度下降方法识别为应用于相应梯度流的优化 - 离散化方案。