We present TabPFN, a trained Transformer that can do supervised
classification for small tabular datasets in less than a second, needs no
hyperparameter tuning and is competitive with state-of-the-art classification
methods. TabPFN is fully entailed in the weights of our network, which accepts
training and test samples as a set-valued input and yields predictions for the
entire test set in a single forward pass. TabPFN is a Prior-Data Fitted Network
(PFN) and is trained offline once, to approximate Bayesian inference on
synthetic datasets drawn from our prior. This prior incorporates ideas from
causal reasoning: It entails a large space of structural causal models with a
preference for simple structures. On the 18 datasets in the OpenML-CC18 suite
that contain up to 1 000 training data points, up to 100 purely numerical
features without missing values, and up to 10 classes, we show that our method
clearly outperforms boosted trees and performs on par with complex
state-of-the-art AutoML systems with up to 70$\times$ speedup. This increases
to a 3200$\times$ speedup when a GPU is available. We also validate these
results on an additional 67 small numerical datasets from OpenML. We provide
all our code, the trained TabPFN, an interactive browser demo and a Colab
notebook at this https URL

TabPFN 是一个训练有素的 Transformer，可对小型表格数据集进行监督分类，无需调整超参数，并且在与最先进的分类方法竞争中具有竞争力，它彻底包含在我们网络的权重中，并接受训练和测试样本以及一系列值输入，并在单个前向传递中为整个测试集产生预测，其先验扩展了因果推理的一些想法，适用于预测与给定输入和输出相关的结构因果模型。