Can we build a single large model for a wide range of PDE-related scientific
learning tasks? Can this model generalize to new PDEs, even of new forms,
without any fine-tuning? In-context operator learning and the corresponding
model In-Context Operator Networks (ICON) [1] represent an initial exploration
of these questions. The capability of ICON regarding the first question has
been demonstrated in [1]. In this paper, we explore the second question by
investigating the generalization capabilities of ICON for conservation laws, a
family of PDEs with temporal evolution. We show the positive answer to the
second question, i.e., ICON can generalize well to some PDEs with new forms
without any fine-tuning. We also show how to broaden the range of problems that
ICON can address, by transforming functions and equations to ICON's capability
scope. We believe that the progress in this paper is a significant step towards
the goal of training a foundation model for PDE-related tasks under the
in-context operator learning framework.

在本文中，我们通过研究 ICON 对守恒定律的广义化能力，探讨了建立一个能够广泛应用于各种 PDE 相关科学学习任务的单个大型模型的可能性，而且这个模型即使对于新形式的 PDE 也不需要经过微调即可广义化。

一维标量非线性守恒方程中基于上下文算子网络的 PDE 推广研究

PDE Generalization of In-Context Operator Networks: A Study on 1D Scalar  Nonlinear Conservation Laws

In the growing domain of scientific machine learning, in-context operator
learning has demonstrated notable potential in learning operators from prompted
data during inference stage without weight updates. However, the current
model's overdependence on sensor data, may inadvertently overlook the
invaluable human insight into the operator. To address this, we present a
transformation of in-context operator learning into a multi-modal paradigm. We
propose the use of "captions" to integrate human knowledge about the operator,
expressed through natural language descriptions and equations. We illustrate
how this method not only broadens the flexibility and generality of
physics-informed learning, but also significantly boosts learning performance
and reduces data needs. Furthermore, we introduce a more efficient neural
network architecture for multi-modal in-context operator learning, referred to
as "ICON-LM", based on a language-model-like architecture. We demonstrate the
viability of "ICON-LM" for scientific machine learning tasks, which creates a
new path for the application of language models.

在科学机器学习的不断发展中，上下文操作符学习在推理阶段从提示数据中学习操作符而无需进行权重更新方面显示出显著潜力。然而，当前模型对传感器数据的过度依赖可能会无意中忽视对操作符的宝贵人类洞察力。为了解决这个问题，我们将上下文操作符学习转化为多模态范式的方法。我们提出使用 “标题” 来集成人类关于操作符的知识，通过自然语言描述和方程式来表达。我们说明了这种方法不仅扩展了物理驱动学习的灵活性和广泛性，而且显著提高了学习性能并减少了数据需求。此外，我们介绍了一种更高效的多模态上下文操作符学习神经网络架构，称为 “ICON-LM”，基于类似语言模型的架构。我们展示了 “ICON-LM” 在科学机器学习任务中的可行性，为语言模型的应用开辟了新的道路。