We introduce multiple physics pretraining (MPP), an autoregressive
task-agnostic pretraining approach for physical surrogate modeling. MPP
involves training large surrogate models to predict the dynamics of multiple
heterogeneous physical systems simultaneously by learning features that are
broadly useful across diverse physical tasks. In order to learn effectively in
this setting, we introduce a shared embedding and normalization strategy that
projects the fields of multiple systems into a single shared embedding space.
We validate the efficacy of our approach on both pretraining and downstream
tasks over a broad fluid mechanics-oriented benchmark. We show that a single
MPP-pretrained transformer is able to match or outperform task-specific
baselines on all pretraining sub-tasks without the need for finetuning. For
downstream tasks, we demonstrate that finetuning MPP-trained models results in
more accurate predictions across multiple time-steps on new physics compared to
training from scratch or finetuning pretrained video foundation models. We
open-source our code and model weights trained at multiple scales for
reproducibility and community experimentation.

我们介绍了一种多物理预训练（MPP）方法，这是一种用于物理代理建模的自回归通用预训练方法。MPP 通过学习在不同物理任务中广泛适用的特征，训练大型代理模型来同时预测多个异质物理系统的动力学。我们引入了共享的嵌入和归一化策略，将多个系统的字段投影到单一共享的嵌入空间中，以有效地学习。我们在广泛的流体力学基准测试中验证了我们方法的有效性，既在预训练任务上，也在下游任务上表现出优于特定任务的基准模型的能力，且无需微调。对于下游任务，我们证明微调 MPP 训练的模型相比从头训练或微调预训练视频基础模型，对多个时间步长上的新物理现象具有更准确的预测能力。我们开源了我们的代码和多个尺度上训练的模型权重，以便复现和社区实验。

物理替代模型的多物理预训练

Multiple Physics Pretraining for Physical Surrogate Models

We propose a method for unsupervised domain adaptation that trains a shared
embedding to align the joint distributions of inputs (domain) and outputs
(classes), making any classifier agnostic to the domain. Joint alignment
ensures that not only the marginal distributions of the domain are aligned, but
the labels as well. We propose a novel objective function that encourages the
class-conditional distributions to have disjoint support in feature space. We
further exploit adversarial regularization to improve the performance of the
classifier on the domain for which no annotated data is available.

本研究提出了一种无监督领域自适应方法，通过训练共享嵌入来对齐输入（域）和输出（类）的联合分布，从而使任何分类器对域都不具有特异性。联合对齐不仅确保域的边际分布对齐，还确保标签对齐，并提出了一种新颖的目标函数，鼓励类条件分布在特征空间具有不相交的支持。此外还可以利用对抗正则化来改进没有注释的域上分类器的性能。