Benchmarks that closely reflect downstream application scenarios are
essential for the streamlined adoption of new research in tabular machine
learning (ML). In this work, we examine existing tabular benchmarks and find
two common characteristics of industry-grade tabular data that are
underrepresented in the datasets available to the academic community. First,
tabular data often changes over time in real-world deployment scenarios. This
impacts model performance and requires time-based train and test splits for
correct model evaluation. Yet, existing academic tabular datasets often lack
timestamp metadata to enable such evaluation. Second, a considerable portion of
datasets in production settings stem from extensive data acquisition and
feature engineering pipelines. For each specific dataset, this can have a
different impact on the absolute and relative number of predictive,
uninformative, and correlated features, which in turn can affect model
selection. To fill the aforementioned gaps in academic benchmarks, we introduce
TabReD -- a collection of eight industry-grade tabular datasets covering a wide
range of domains from finance to food delivery services. We assess a large
number of tabular ML models in the feature-rich, temporally-evolving data
setting facilitated by TabReD. We demonstrate that evaluation on time-based
data splits leads to different methods ranking, compared to evaluation on
random splits more common in academic benchmarks. Furthermore, on the TabReD
datasets, MLP-like architectures and GBDT show the best results, while more
sophisticated DL models are yet to prove their effectiveness.

本文研究了标准机器学习中现成的表格式基准数据集存在的问题，并引入了 TabReD 数据集以填补学术基准数据集的不足之处，以此评估不同 ML 模型在时间演化的数据环境中的表现。结果表明，基于时间的数据划分与学术基准数据集中的随机划分会导致不同的方法排名，MLP 类的架构和 GBDT 模型在 TabReD 数据集上展现出最佳结果，而更复杂的深度学习模型还需进一步证明其有效性。