Automated machine learning (AutoML) can produce complex model ensembles by stacking, bagging, and boosting many individual models like trees, deep networks, and nearest neighbor estimators. While highly accurate, the resulting predictors are large, slow, and opaque as compared to their constituents. To improve the deployment of AutoML on tabular data, we propose FAST-DAD to distill arbitrarily complex ensemble predictors into individual models like boosted trees, random forests, and deep networks. At the heart of our approach is a data augmentation strategy based on Gibbs sampling from a self-attention pseudolikelihood estimator. Across 30 datasets spanning regression and binary/multiclass classification tasks, FAST-DAD distillation produces significantly better individual models than one obtains through standard training on the original data. Our individual distilled models are over 10x faster and more accurate than ensemble predictors produced by AutoML tools like H2O/AutoSklearn.

FAST-DAD是一种能够将复杂的模型集合精简为促进部署的个体模型的方法，它基于数据增强策略，并使用基于自注意力的伪似然估计器的吉布斯采样。在30个跨越回归和二元/多类分类任务的数据集上，FAST-DAD精炼方法得到的个体模型比在原始数据上进行标准培训获得的模型更快而且更准确，可以超过AutoML工具如H2O / AutoSklearn生成的集成预测器10倍以上。

增强蒸馏技术的表格数据快速准确简单模型