Experimental design is a fundamental problem in many science and engineering
fields. In this problem, sample efficiency is crucial due to the time, money,
and safety costs of real-world design evaluations. Existing approaches either
rely on active data collection or access to large, labeled datasets of past
experiments, making them impractical in many real-world scenarios. In this
work, we address the more challenging yet realistic setting of few-shot
experimental design, where only a few labeled data points of input designs and
their corresponding values are available. We approach this problem as a
conditional generation task, where a model conditions on a few labeled examples
and the desired output to generate an optimal input design. To this end, we
introduce Experiment Pretrained Transformers (ExPT), a foundation model for
few-shot experimental design that employs a novel combination of synthetic
pretraining with in-context learning. In ExPT, we only assume knowledge of a
finite collection of unlabelled data points from the input domain and pretrain
a transformer neural network to optimize diverse synthetic functions defined
over this domain. Unsupervised pretraining allows ExPT to adapt to any design
task at test time in an in-context fashion by conditioning on a few labeled
data points from the target task and generating the candidate optima. We
evaluate ExPT on few-shot experimental design in challenging domains and
demonstrate its superior generality and performance compared to existing
methods. The source code is available at this https URL

在少样本条件下进行实验设计，ExPT 模型是一个用于少样本实验设计的基础模型，采用了合成预训练和上下文学习的新型组合，通过以目标任务的少量标记数据点为条件生成候选最优结果，从而适应任何设计任务并展现出优越的普适性和性能。