Foundation models are multi-dataset and multi-task machine learning methods
that once pre-trained can be fine-tuned for a large variety of downstream
applications. The successful development of such general-purpose models for
physics data would be a major breakthrough as they could improve the achievable
physics performance while at the same time drastically reduce the required
amount of training time and data.
We report significant progress on this challenge on several fronts. First, a
comprehensive set of evaluation methods is introduced to judge the quality of
an encoding from physics data into a representation suitable for the
autoregressive generation of particle jets with transformer architectures (the
common backbone of foundation models). These measures motivate the choice of a
higher-fidelity tokenization compared to previous works. Finally, we
demonstrate transfer learning between an unsupervised problem (jet generation)
and a classic supervised task (jet tagging) with our new OmniJet-$\alpha$
model. This is the first successful transfer between two different and actively
studied classes of tasks and constitutes a major step in the building of
foundation models for particle physics.

多数据集和多任务的基础模型能够在预先训练后为各种下游应用进行微调，本文在物理数据上取得重大突破，引入了一套评估方法，并展示了无监督问题和经典监督任务之间的成功迁移学习，这是粒子物理学基础模型构建的重要一步。

OmniJet-$α$: 跨任务的粒子物理学基础模型

OmniJet-$α$: The first cross-task foundation model for particle  physics

Observing the close relationship among panoptic, semantic and instance
segmentation tasks, we propose to train a universal multi-dataset multi-task
segmentation model: DaTaSeg.We use a shared representation (mask proposals with
class predictions) for all tasks. To tackle task discrepancy, we adopt
different merge operations and post-processing for different tasks. We also
leverage weak-supervision, allowing our segmentation model to benefit from
cheaper bounding box annotations. To share knowledge across datasets, we use
text embeddings from the same semantic embedding space as classifiers and share
all network parameters among datasets. We train DaTaSeg on ADE semantic, COCO
panoptic, and Objects365 detection datasets. DaTaSeg improves performance on
all datasets, especially small-scale datasets, achieving 54.0 mIoU on ADE
semantic and 53.5 PQ on COCO panoptic. DaTaSeg also enables weakly-supervised
knowledge transfer on ADE panoptic and Objects365 instance segmentation.
Experiments show DaTaSeg scales with the number of training datasets and
enables open-vocabulary segmentation through direct transfer. In addition, we
annotate an Objects365 instance segmentation set of 1,000 images and will
release it as a public benchmark.

提出了一种称为 DaTaSeg 的通用多数据集多任务分割模型，通过使用共享表示，并采用不同的合并操作和后处理，利用弱监督和共享知识的方法，在 ADE 语义、COCO 全景和 Objects365 检测数据集上进行了训练，并能够实现直接转移的开放词汇分割。