We introduce a novel two-level overlapping additive Schwarz preconditioner
for accelerating the training of scientific machine learning applications. The
design of the proposed preconditioner is motivated by the nonlinear two-level
overlapping additive Schwarz preconditioner. The neural network parameters are
decomposed into groups (subdomains) with overlapping regions. In addition, the
network's feed-forward structure is indirectly imposed through a novel
subdomain-wise synchronization strategy and a coarse-level training step.
Through a series of numerical experiments, which consider physics-informed
neural networks and operator learning approaches, we demonstrate that the
proposed two-level preconditioner significantly speeds up the convergence of
the standard (LBFGS) optimizer while also yielding more accurate machine
learning models. Moreover, the devised preconditioner is designed to take
advantage of model-parallel computations, which can further reduce the training
time.

提出了一种新颖的双层重叠添加 Schwarz 预条件器，用于加速科学机器学习应用的训练，通过在网络参数中引入重叠区域的组（子域），间接通过一种新颖的子域同步策略和粗粒度训练步骤实现网络的前馈结构，并通过一系列数值实验表明，所提出的双层预条件器显著加速了标准（LBFGS）优化器的收敛速度，同时产生了更准确的机器学习模型，此外，所设计的预条件器可以充分利用模型并行计算，从而进一步减少训练时间。