Forecasting high-dimensional dynamical systems is a fundamental challenge in
various fields, such as the geosciences and engineering. Neural Ordinary
Differential Equations (NODEs), which combine the power of neural networks and
numerical solvers, have emerged as a promising algorithm for forecasting
complex nonlinear dynamical systems. However, classical techniques used for
NODE training are ineffective for learning chaotic dynamical systems. In this
work, we propose a novel NODE-training approach that allows for robust learning
of chaotic dynamical systems. Our method addresses the challenges of
non-convexity and exploding gradients associated with underlying chaotic
dynamics. Training data trajectories from such systems are split into multiple,
non-overlapping time windows. In addition to the deviation from the training
data, the optimization loss term further penalizes the discontinuities of the
predicted trajectory between the time windows. The window size is selected
based on the fastest Lyapunov time scale of the system. Multi-step penalty(MP)
method is first demonstrated on Lorenz equation, to illustrate how it improves
the loss landscape and thereby accelerating the optimization convergence. MP
method can optimize chaotic systems in a manner similar to least-squares
shadowing with significantly lower computational costs. Our proposed algorithm,
denoted the Multistep Penalty NODE(MP-NODE), is applied to chaotic systems such
as the Kuramoto-Sivashinsky equation and the two-dimensional Kolmogorov flow.
It is observed that MP-NODE provide viable performance for such chaotic
systems, not only for short-term trajectory predictions but also for invariant
statistics that are hallmarks of the chaotic nature of these dynamics.

提出了一种新颖的多步惩罚 NODE（MP-NODE）算法，可以强化学习混沌动力学系统，并显示在短期轨迹预测和代表这些动力学混沌特性的不变统计方面具有可行的性能。

分而治之：学习具有多步惩罚的混沌动力系统的神经常微分方程

Divide And Conquer: Learning Chaotic Dynamical Systems With Multistep  Penalty Neural Ordinary Differential Equations

Diffusion models are a class of probabilistic generative models that have
been widely used as a prior for image processing tasks like text conditional
generation and inpainting. We demonstrate that these models can be adapted to
make predictions and provide uncertainty quantification for chaotic dynamical
systems. In these applications, diffusion models can implicitly represent
knowledge about outliers and extreme events; however, querying that knowledge
through conditional sampling or measuring probabilities is surprisingly
difficult. Existing methods for conditional sampling at inference time seek
mainly to enforce the constraints, which is insufficient to match the
statistics of the distribution or compute the probability of the chosen events.
To achieve these ends, optimally one would use the conditional score function,
but its computation is typically intractable. In this work, we develop a
probabilistic approximation scheme for the conditional score function which
provably converges to the true distribution as the noise level decreases. With
this scheme we are able to sample conditionally on nonlinear userdefined events
at inference time, and matches data statistics even when sampling from the
tails of the distribution.

通过开发一种概率近似方案，使扩散模型能够在推理时间内有条件地采样非线性用户定义事件，从而匹配数据统计，即使从分布的尾部进行采样。

物理动力系统扩散模型中的用户定义事件采样与不确定性量化

User-defined Event Sampling and Uncertainty Quantification in Diffusion  Models for Physical Dynamical Systems

Chaotic dynamical systems (DS) are ubiquitous in nature and society. Often we
are interested in reconstructing such systems from observed time series for
prediction or mechanistic insight, where by reconstruction we mean learning
geometrical and invariant temporal properties of the system in question (like
attractors). However, training reconstruction algorithms like recurrent neural
networks (RNNs) on such systems by gradient-descent based techniques faces
severe challenges. This is mainly due to exploding gradients caused by the
exponential divergence of trajectories in chaotic systems. Moreover, for
(scientific) interpretability we wish to have as low dimensional
reconstructions as possible, preferably in a model which is mathematically
tractable. Here we report that a surprisingly simple modification of teacher
forcing leads to provably strictly all-time bounded gradients in training on
chaotic systems, and, when paired with a simple architectural rearrangement of
a tractable RNN design, piecewise-linear RNNs (PLRNNs), allows for faithful
reconstruction in spaces of at most the dimensionality of the observed system.
We show on several DS that with these amendments we can reconstruct DS better
than current SOTA algorithms, in much lower dimensions. Performance differences
were particularly compelling on real world data with which most other methods
severely struggled. This work thus led to a simple yet powerful DS
reconstruction algorithm which is highly interpretable at the same time.

通过对混沌动力系统进行简单的修改以及与可处理的递归神经网络设计的简单架构重排，可以在低维空间中准确还原动力系统，并在真实世界数据上表现出较高的性能。

广义教师强制学习混沌动力学

Generalized Teacher Forcing for Learning Chaotic Dynamics

Machine Learning (ML) inspired algorithms provide a flexible set of tools for
analyzing and forecasting chaotic dynamical systems. We here analyze the
performance of one algorithm for the prediction of extreme events in the
two-dimensional H\'enon map at the classical parameters. The task is to
determine whether a trajectory will exceed a threshold after a set number of
time steps into the future. This task has a geometric interpretation within the
dynamics of the H\'enon map, which we use to gauge the performance of the
neural networks that are used in this work. We analyze the dependence of the
success rate of the ML models on the prediction time $T$ , the number of
training samples $N_T$ and the size of the network $N_p$. We observe that in
order to maintain a certain accuracy, $N_T \propto exp(2 h T)$ and $N_p \propto
exp(hT)$, where $h$ is the topological entropy. Similar relations between the
intrinsic chaotic properties of the dynamics and ML parameters might be
observable in other systems as well.

通过分析经典参数下二维 Henon 地图中的一个算法，在预测极端事件方面确定算法的性能，并发现在保持一定精度的情况下，机器学习算法的成功率依赖于预测时间、训练样本数和网络规模等因素，其内在混沌性质和机器学习参数之间存在类似的关系。