While physics-informed neural networks (PINNs) have become a popular deep
learning framework for tackling forward and inverse problems governed by
partial differential equations (PDEs), their performance is known to degrade
when larger and deeper neural network architectures are employed. Our study
identifies that the root of this counter-intuitive behavior lies in the use of
multi-layer perceptron (MLP) architectures with non-suitable initialization
schemes, which result in poor trainablity for the network derivatives, and
ultimately lead to an unstable minimization of the PDE residual loss. To
address this, we introduce Physics-informed Residual Adaptive Networks
(PirateNets), a novel architecture that is designed to facilitate stable and
efficient training of deep PINN models. PirateNets leverage a novel adaptive
residual connection, which allows the networks to be initialized as shallow
networks that progressively deepen during training. We also show that the
proposed initialization scheme allows us to encode appropriate inductive biases
corresponding to a given PDE system into the network architecture. We provide
comprehensive empirical evidence showing that PirateNets are easier to optimize
and can gain accuracy from considerably increased depth, ultimately achieving
state-of-the-art results across various benchmarks. All code and data
accompanying this manuscript will be made publicly available at
https://github.com/PredictiveIntelligenceLab/jaxpi.

该研究针对物理信息神经网络（PINNs）在使用大型深度神经网络架构处理偏微分方程（PDE）驱动的正向和反向问题时性能下降的问题进行了研究，发现这种行为的根源是采用了不适合的初始化方案，导致网络对导数的训练能力较差，最终导致 PDE 残差损失的不稳定最小化。为了解决这个问题，引入了一种名为 Physics-informed Residual Adaptive Networks（PirateNets）的新型架构，旨在实现深层 PINN 模型的稳定和高效训练。PirateNets 采用一种新颖的自适应残差连接，允许网络在训练过程中作为浅层网络逐渐加深。研究还表明，所提出的初始化方案可以将给定 PDE 系统的适当归纳偏差编码到网络架构中。通过全面的实证研究，证明了 PirateNets 更易优化，并且可以从较大深度中获得准确性，最终在各种基准测试中取得了最先进的结果。本文附带的代码和数据将在 https://github.com/PredictiveIntelligenceLab/jaxpi 公开提供。