Solving partial differential equations (PDEs) numerically often requires huge
computing time, energy cost, and hardware resources in practical applications.
This has limited their applications in many scenarios (e.g., autonomous
systems, supersonic flows) that have a limited energy budget and require near
real-time response. Leveraging optical computing, this paper develops an
on-chip training framework for physics-informed neural networks (PINNs), aiming
to solve high-dimensional PDEs with fJ/MAC photonic power consumption and
ultra-low latency. Despite the ultra-high speed of optical neural networks,
training a PINN on an optical chip is hard due to (1) the large size of
photonic devices, and (2) the lack of scalable optical memory devices to store
the intermediate results of back-propagation (BP). To enable realistic optical
PINN training, this paper presents a scalable method to avoid the BP process.
We also employ a tensor-compressed approach to improve the convergence and
scalability of our optical PINN training. This training framework is designed
with tensorized optical neural networks (TONN) for scalable inference
acceleration and MZI phase-domain tuning for \textit{in-situ} optimization. Our
simulation results of a 20-dim HJB PDE show that our photonic accelerator can
reduce the number of MZIs by a factor of $1.17\times 10^3$, with only $1.36$ J
and $1.15$ s to solve this equation. This is the first real-size optical PINN
training framework that can be applied to solve high-dimensional PDEs.

光学计算在这篇论文中发挥了重要作用，提出了一个基于芯片的物理信息神经网络（PINNs）训练框架，用于高维偏微分方程的求解，具有相当低的能量消耗和超低的延迟。

实时 FJ/MAC PDE 求解器通过张量化的非反向传播光学 PINN 训练

Real-Time FJ/MAC PDE Solvers via Tensorized, Back-Propagation-Free  Optical PINN Training

Physics-Informed Neural Networks (PINNs) have proven effective in solving
partial differential equations (PDEs), especially when some data are available
by blending seamlessly data and physics. However, extending PINNs to
high-dimensional and even high-order PDEs encounters significant challenges due
to the computational cost associated with automatic differentiation in the
residual loss. Herein, we address the limitations of PINNs in handling
high-dimensional and high-order PDEs by introducing Hutchinson Trace Estimation
(HTE). Starting with the second-order high-dimensional PDEs ubiquitous in
scientific computing, HTE transforms the calculation of the entire Hessian
matrix into a Hessian vector product (HVP). This approach alleviates the
computational bottleneck via Taylor-mode automatic differentiation and
significantly reduces memory consumption from the Hessian matrix to HVP. We
further showcase HTE's convergence to the original PINN loss and its unbiased
behavior under specific conditions. Comparisons with Stochastic Dimension
Gradient Descent (SDGD) highlight the distinct advantages of HTE, particularly
in scenarios with significant variance among dimensions. We further extend HTE
to higher-order and higher-dimensional PDEs, specifically addressing the
biharmonic equation. By employing tensor-vector products (TVP), HTE efficiently
computes the colossal tensor associated with the fourth-order high-dimensional
biharmonic equation, saving memory and enabling rapid computation. The
effectiveness of HTE is illustrated through experimental setups, demonstrating
comparable convergence rates with SDGD under memory and speed constraints.
Additionally, HTE proves valuable in accelerating the Gradient-Enhanced PINN
(gPINN) version as well as the Biharmonic equation. Overall, HTE opens up a new
capability in scientific machine learning for tackling high-order and
high-dimensional PDEs.

通过引入 Hutchinson Trace Estimation (HTE)，我们解决了 PINNs 在处理高维和高阶偏微分方程方面的局限性，通过将计算整个 Hessian 矩阵转化为 Hessian 向量积 (HVP) 的方式，通过 Taylor 模态自动微分减轻了计算瓶颈，并将内存消耗显著减少，进一步通过与 Stochastic Dimension Gradient Descent (SDGD) 的比较突显了 HTE 的明显优势，特别是在维度之间存在较大方差的情况下。

用于高维度和高阶物理信息神经网络的 Hutchinson 迹估计

Hutchinson Trace Estimation for High-Dimensional and High-Order  Physics-Informed Neural Networks

The curse-of-dimensionality (CoD) taxes computational resources heavily with
exponentially increasing computational cost as the dimension increases. This
poses great challenges in solving high-dimensional PDEs as Richard Bellman
first pointed out over 60 years ago. While there has been some recent success
in solving numerically partial differential equations (PDEs) in high
dimensions, such computations are prohibitively expensive, and true scaling of
general nonlinear PDEs to high dimensions has never been achieved. In this
paper, we develop a new method of scaling up physics-informed neural networks
(PINNs) to solve arbitrary high-dimensional PDEs. The new method, called
Stochastic Dimension Gradient Descent (SDGD), decomposes a gradient of PDEs
into pieces corresponding to different dimensions and samples randomly a subset
of these dimensional pieces in each iteration of training PINNs. We
theoretically prove the convergence guarantee and other desired properties of
the proposed method. We experimentally demonstrate that the proposed method
allows us to solve many notoriously hard high-dimensional PDEs, including the
Hamilton-Jacobi-Bellman and the Schr\"{o}dinger equations in thousands of
dimensions very fast on a single GPU using the PINNs mesh-free approach. For
example, we solve nontrivial nonlinear PDEs (the HJB-Lin equation and the BSB
equation) in 100,000 dimensions in 6 hours on a single GPU using SDGD with
PINNs. Since SDGD is a general training methodology of PINNs, SDGD can be
applied to any current and future variants of PINNs to scale them up for
arbitrary high-dimensional PDEs.

开发了一种名为 Stochastic Dimension Gradient Descent (SDGD) 的新方法，用于扩展物理信息神经网络（PINNs）以解决任意高维 PDEs，并理论上证明了其收敛性和其他所需属性。

利用物理知识驱动的神经网络解决维度灾难

Tackling the Curse of Dimensionality with Physics-Informed Neural  Networks

High-dimensional PDEs have been a longstanding computational challenge. We
propose to solve high-dimensional PDEs by approximating the solution with a
deep neural network which is trained to satisfy the differential operator,
initial condition, and boundary conditions. Our algorithm is meshfree, which is
key since meshes become infeasible in higher dimensions. Instead of forming a
mesh, the neural network is trained on batches of randomly sampled time and
space points. The algorithm is tested on a class of high-dimensional free
boundary PDEs, which we are able to accurately solve in up to $200$ dimensions.
The algorithm is also tested on a high-dimensional Hamilton-Jacobi-Bellman PDE
and Burgers' equation. The deep learning algorithm approximates the general
solution to the Burgers' equation for a continuum of different boundary
conditions and physical conditions (which can be viewed as a high-dimensional
space). We call the algorithm a "Deep Galerkin Method (DGM)" since it is
similar in spirit to Galerkin methods, with the solution approximated by a
neural network instead of a linear combination of basis functions. In addition,
we prove a theorem regarding the approximation power of neural networks for a
class of quasilinear parabolic PDEs.

提出了一种名为 Deep Galerkin Method（DGM）的算法，它使用深度神经网络来解决高维偏微分方程问题。相较于形成网格，该算法不依赖于网格，而是通过对随机采样的时间和空间点的批量训练来实现。它在一类高维自由边界的偏微分方程、高维哈密顿 - 雅各比 - 贝尔曼（Hamilton-Jacobi-Bellman）偏微分方程和 Burgers 方程上得到了测试，并且在高维空间中能够准确地近似各种边界条件和物理条件下的 Burgers 方程的一般解。此外，论文还证明了神经网络在一类拟线性抛物型偏微分方程上的逼近能力。