Recent work in scientific machine learning has developed so-called
physics-informed neural network (PINN) models. The typical approach is to
incorporate physical domain knowledge as soft constraints on an empirical loss
function and use existing machine learning methodologies to train the model. We
demonstrate that, while existing PINN methodologies can learn good models for
relatively trivial problems, they can easily fail to learn relevant physical
phenomena for even slightly more complex problems. In particular, we analyze
several distinct situations of widespread physical interest, including learning
differential equations with convection, reaction, and diffusion operators. We
provide evidence that the soft regularization in PINNs, which involves
PDE-based differential operators, can introduce a number of subtle problems,
including making the problem more ill-conditioned. Importantly, we show that
these possible failure modes are not due to the lack of expressivity in the NN
architecture, but that the PINN's setup makes the loss landscape very hard to
optimize. We then describe two promising solutions to address these failure
modes. The first approach is to use curriculum regularization, where the PINN's
loss term starts from a simple PDE regularization, and becomes progressively
more complex as the NN gets trained. The second approach is to pose the problem
as a sequence-to-sequence learning task, rather than learning to predict the
entire space-time at once. Extensive testing shows that we can achieve up to
1-2 orders of magnitude lower error with these methods as compared to regular
PINN training.

本文研究了物理知识对神经网络的影响，尤其是对物理意义的学习。研究发现，使用以前的方法，神经网络会容易受到微妙的问题的困扰。为了解决这个问题，我们提出了课程规范化和序列到序列学习两种新的方法。通过使用这两种方法，我们可以取得比以前更好的结果。