The residual loss in Physics-Informed Neural Networks (PINNs) alters the
simple recursive relation of layers in a feed-forward neural network by
applying a differential operator, resulting in a loss landscape that is
inherently different from those of common supervised problems. Therefore,
relying on the existing theory leads to unjustified design choices and
suboptimal performance. In this work, we analyze the residual loss by studying
its characteristics at critical points to find the conditions that result in
effective training of PINNs. Specifically, we first show that under certain
conditions, the residual loss of PINNs can be globally minimized by a wide
neural network. Furthermore, our analysis also reveals that an activation
function with well-behaved high-order derivatives plays a crucial role in
minimizing the residual loss. In particular, to solve a $k$-th order PDE, the
$k$-th derivative of the activation function should be bijective. The
established theory paves the way for designing and choosing effective
activation functions for PINNs and explains why periodic activations have shown
promising performance in certain cases. Finally, we verify our findings by
conducting a set of experiments on several PDEs. Our code is publicly available
at this https URL

本研究分析了物理信息神经网络（PINNs）中的剩余损失，并发现了通过研究其在临界点处的特性，找到了实现有效训练的条件。我们的分析揭示了一个良好高阶导数的激活函数在最小化剩余损失中起关键作用，进而提供了设计和选择 PINNs 有效激活函数的理论依据，并解释了为什么周期性激活函数在某些情况下表现出有希望的性能。最后，通过在几个偏微分方程上进行一系列实验验证了我们的发现。