Recent years have witnessed the promise of coupling machine learning methods and physical domain-specific insight for solving scientific problems based on partial differential equations (PDEs). However, being data-intensive, these methods still require a large amount of PDE data. This reintroduces the need for expensive numerical PDE solutions, partially undermining the original goal of avoiding these expensive simulations. In this work, seeking data efficiency, we design unsupervised pretraining and in-context learning methods for PDE operator learning. To reduce the need for training data with simulated solutions, we pretrain neural operators on unlabeled PDE data using reconstruction-based proxy tasks. To improve out-of-distribution performance, we further assist neural operators in flexibly leveraging in-context learning methods, without incurring extra training costs or designs. Extensive empirical evaluations on a diverse set of PDEs demonstrate that our method is highly data-efficient, more generalizable, and even outperforms conventional vision-pretrained models.

通过机器学习方法和物理领域专业见解相结合，解决基于偏微分方程的科学问题在近年来取得了很大的进展。然而，这些方法仍然需要大量的偏微分方程数据。为了提高数据效率，我们设计了无监督的预训练和上下文学习方法用于偏微分方程算子学习，通过重构为代理任务的无标签偏微分方程数据对神经算子进行预训练。为了提高超出分布的性能，我们进一步辅助神经算子以灵活地利用上下文学习方法，而无需额外的训练成本或设计。对多种偏微分方程进行的大量实证评估表明我们的方法具有高度的数据效率、更好的泛化性能，甚至胜过传统的预训练模型。

通过无监督预训练和上下文学习实现高效的运算符学习