A primary challenge of physics-informed machine learning (PIML) is its
generalization beyond the training domain, especially when dealing with complex
physical problems represented by partial differential equations (PDEs). This
paper aims to enhance the generalization capabilities of PIML, facilitating
practical, real-world applications where accurate predictions in unexplored
regions are crucial. We leverage the inherent causality and temporal sequential
characteristics of PDE solutions to fuse PIML models with recurrent neural
architectures based on systems of ordinary differential equations, referred to
as neural oscillators. Through effectively capturing long-time dependencies and
mitigating the exploding and vanishing gradient problem, neural oscillators
foster improved generalization in PIML tasks. Extensive experimentation
involving time-dependent nonlinear PDEs and biharmonic beam equations
demonstrates the efficacy of the proposed approach. Incorporating neural
oscillators outperforms existing state-of-the-art methods on benchmark problems
across various metrics. Consequently, the proposed method improves the
generalization capabilities of PIML, providing accurate solutions for
extrapolation and prediction beyond the training data.

该研究提出了一种新方法，通过融合系统常微分方程所基于的神经振荡器，有效地捕捉长期依赖关系并解决爆炸梯度问题，从而增强了物理先验机器学习模型在复杂物理问题中的泛化能力。通过在时间依赖的非线性偏微分方程和双调和梁方程中进行实验，证明了该方法在基准问题上的优越性能，显著提高了物理先验机器学习的泛化能力，为训练数据之外的外推和预测提供了准确的解决方案。

神经振荡器用于物理推断机器学习的泛化

Neural oscillators for generalization of physics-informed machine  learning

Coupled oscillators are being increasingly used as the basis of machine
learning (ML) architectures, for instance in sequence modeling, graph
representation learning and in physical neural networks that are used in analog
ML devices. We introduce an abstract class of neural oscillators that
encompasses these architectures and prove that neural oscillators are
universal, i.e, they can approximate any continuous and casual operator mapping
between time-varying functions, to desired accuracy. This universality result
provides theoretical justification for the use of oscillator based ML systems.
The proof builds on a fundamental result of independent interest, which shows
that a combination of forced harmonic oscillators with a nonlinear read-out
suffices to approximate the underlying operators.

介绍了一种抽象的神经振荡器类，证明了神经振荡器是通用的，并可近似于任何时间变化函数之间的映射。这个证明为使用基于振荡器的 ML 系统提供了理论依据。

神经振荡器是普适的

Neural Oscillators are Universal

Circuits of biological neurons, such as in the functional parts of the brain
can be modeled as networks of coupled oscillators. Inspired by the ability of
these systems to express a rich set of outputs while keeping (gradients of)
state variables bounded, we propose a novel architecture for recurrent neural
networks. Our proposed RNN is based on a time-discretization of a system of
second-order ordinary differential equations, modeling networks of controlled
nonlinear oscillators. We prove precise bounds on the gradients of the hidden
states, leading to the mitigation of the exploding and vanishing gradient
problem for this RNN. Experiments show that the proposed RNN is comparable in
performance to the state of the art on a variety of benchmarks, demonstrating
the potential of this architecture to provide stable and accurate RNNs for
processing complex sequential data.

本文中提出基于神经振荡器非线性控制网络的二阶常微分方程模型时间离散化的递归神经网络，证明其对于隐藏状态的梯度具有精确的上限，解决了梯度问题，实验证明该递归神经网络在处理复杂的时序数据上具有稳定和准确的性能，达到了现有技术水平。