Neural operator learning models have emerged as very effective surrogates in
data-driven methods for partial differential equations (PDEs) across different
applications from computational science and engineering. Such operator learning
models not only predict particular instances of a physical or biological system
in real-time but also forecast classes of solutions corresponding to a
distribution of initial and boundary conditions or forcing terms. % DeepONet is
the first neural operator model and has been tested extensively for a broad
class of solutions, including Riemann problems. Transformers have not been used
in that capacity, and specifically, they have not been tested for solutions of
PDEs with low regularity. %
In this work, we first establish the theoretical groundwork that transformers
possess the universal approximation property as operator learning models.
We then apply transformers to forecast solutions of diverse dynamical systems
with solutions of finite regularity for a plurality of initial conditions and
forcing terms. In particular, we consider three examples: the Izhikevich neuron
model, the tempered fractional-order Leaky Integrate-and-Fire (LIF) model, and
the one-dimensional Euler equation Riemann problem. For the latter problem, we
also compare with variants of DeepONet, and we find that transformers
outperform DeepONet in accuracy but they are computationally more expensive.

神经算子学习模型被证实为部分微分方程在各种应用中的高效代理方法，本文通过建立理论基础将变压器作为算子学习模型实现通用逼近性，并应用于预测具有不同初始条件和强迫项的有限正则性动力学系统的解。