In dynamical systems reconstruction (DSR) we seek to infer from time series
measurements a generative model of the underlying dynamical process. This is a
prime objective in any scientific discipline, where we are particularly
interested in parsimonious models with a low parameter load. A common strategy
here is parameter pruning, removing all parameters with small weights. However,
here we find this strategy does not work for DSR, where even low magnitude
parameters can contribute considerably to the system dynamics. On the other
hand, it is well known that many natural systems which generate complex
dynamics, like the brain or ecological networks, have a sparse topology with
comparatively few links. Inspired by this, we show that geometric pruning,
where in contrast to magnitude-based pruning weights with a low contribution to
an attractor's geometrical structure are removed, indeed manages to reduce
parameter load substantially without significantly hampering DSR quality. We
further find that the networks resulting from geometric pruning have a specific
type of topology, and that this topology, and not the magnitude of weights, is
what is most crucial to performance. We provide an algorithm that automatically
generates such topologies which can be used as priors for generative modeling
of dynamical systems by RNNs, and compare it to other well studied topologies
like small-world or scale-free networks.

动态系统重建中的几何修剪可以减少参数负荷，而不显著影响质量，其结果网络具有特定类型的拓扑结构，优于其他研究中的小世界或无标度网络拓扑。

优化循环网络拓扑结构的动力系统重构

Optimal Recurrent Network Topologies for Dynamical Systems  Reconstruction

In science we are interested in finding the governing equations, the
dynamical rules, underlying empirical phenomena. While traditionally scientific
models are derived through cycles of human insight and experimentation,
recently deep learning (DL) techniques have been advanced to reconstruct
dynamical systems (DS) directly from time series data. State-of-the-art
dynamical systems reconstruction (DSR) methods show promise in capturing
invariant and long-term properties of observed DS, but their ability to
generalize to unobserved domains remains an open challenge. Yet, this is a
crucial property we would expect from any viable scientific theory. In this
work, we provide a formal framework that addresses generalization in DSR. We
explain why and how out-of-domain (OOD) generalization (OODG) in DSR profoundly
differs from OODG considered elsewhere in machine learning. We introduce
mathematical notions based on topological concepts and ergodic theory to
formalize the idea of learnability of a DSR model. We formally prove that
black-box DL techniques, without adequate structural priors, generally will not
be able to learn a generalizing DSR model. We also show this empirically,
considering major classes of DSR algorithms proposed so far, and illustrate
where and why they fail to generalize across the whole phase space. Our study
provides the first comprehensive mathematical treatment of OODG in DSR, and
gives a deeper conceptual understanding of where the fundamental problems in
OODG lie and how they could possibly be addressed in practice.

我们提供了一个形式框架，解决了动力系统重构中的泛化问题。通过引入基于拓扑概念和遍历理论的数学概念，我们证明了黑盒深度学习技术通常无法学习到具有泛化能力的动力系统重构模型。我们的研究对动力系统重构中的泛化问题进行了首次全面的数学处理，并深入理解了泛化问题的根本原因以及如何在实践中解决这些问题。