Data preprocessing is a crucial step in the machine learning process that transforms raw data into a more usable format for downstream ML models. However, it can be costly and time-consuming, often requiring the expertise of domain experts. Existing automated machine learning (AutoML) frameworks claim to automate data preprocessing. However, they often use a restricted search space of data preprocessing pipelines which limits the potential performance gains, and they are often too slow as they require training the ML model multiple times. In this paper, we propose DiffPrep, a method that can automatically and efficiently search for a data preprocessing pipeline for a given tabular dataset and a differentiable ML model such that the performance of the ML model is maximized. We formalize the problem of data preprocessing pipeline search as a bi-level optimization problem. To solve this problem efficiently, we transform and relax the discrete, non-differential search space into a continuous and differentiable one, which allows us to perform the pipeline search using gradient descent with training the ML model only once. Our experiments show that DiffPrep achieves the best test accuracy on 15 out of the 18 real-world datasets evaluated and improves the model's test accuracy by up to 6.6 percentage points.

我们提出了一种名为DiffPrep的方法，它可以自动且高效地搜索适用于给定表格数据集和可微分机器学习模型的数据预处理流水线，以使机器学习模型的性能达到最大化。我们将数据预处理流水线搜索问题形式化为一个双层优化问题，并通过将离散、非可微分的搜索空间转化和放松为连续、可微分的空间来高效求解，从而只需训练一次机器学习模型即可进行流水线搜索。实验结果表明DiffPrep在评估的18个真实数据集中有15个取得了最佳的测试准确率，并将模型的测试准确率提高了多达6.6个百分点。

DiffPrep: 可微分的表格数据预处理流水线搜索