A primary challenge of physics-informed machine learning (PIML) is its
generalization beyond the training domain, especially when dealing with complex
physical problems represented by partial differential equations (PDEs). This
paper aims to enhance the generalization capabilities of PIML, facilitating
practical, real-world applications where accurate predictions in unexplored
regions are crucial. We leverage the inherent causality and temporal sequential
characteristics of PDE solutions to fuse PIML models with recurrent neural
architectures based on systems of ordinary differential equations, referred to
as neural oscillators. Through effectively capturing long-time dependencies and
mitigating the exploding and vanishing gradient problem, neural oscillators
foster improved generalization in PIML tasks. Extensive experimentation
involving time-dependent nonlinear PDEs and biharmonic beam equations
demonstrates the efficacy of the proposed approach. Incorporating neural
oscillators outperforms existing state-of-the-art methods on benchmark problems
across various metrics. Consequently, the proposed method improves the
generalization capabilities of PIML, providing accurate solutions for
extrapolation and prediction beyond the training data.

该研究提出了一种新方法，通过融合系统常微分方程所基于的神经振荡器，有效地捕捉长期依赖关系并解决爆炸梯度问题，从而增强了物理先验机器学习模型在复杂物理问题中的泛化能力。通过在时间依赖的非线性偏微分方程和双调和梁方程中进行实验，证明了该方法在基准问题上的优越性能，显著提高了物理先验机器学习的泛化能力，为训练数据之外的外推和预测提供了准确的解决方案。

神经振荡器用于物理推断机器学习的泛化

Neural oscillators for generalization of physics-informed machine  learning

The design of recurrent neural networks (RNNs) to accurately process
sequential inputs with long-time dependencies is very challenging on account of
the exploding and vanishing gradient problem. To overcome this, we propose a
novel RNN architecture which is based on a structure preserving discretization
of a Hamiltonian system of second-order ordinary differential equations that
models networks of oscillators. The resulting RNN is fast, invertible (in
time), memory efficient and we derive rigorous bounds on the hidden state
gradients to prove the mitigation of the exploding and vanishing gradient
problem. A suite of experiments are presented to demonstrate that the proposed
RNN provides state of the art performance on a variety of learning tasks with
(very) long-time dependencies.

本文提出了一种基于哈密顿系统的离散化的循环神经网络架构，解决长时依赖序列输入处理的梯度消失和爆炸问题，实验表明该方法在各种学习任务中提供了最先进的性能。