Benchmarks that closely reflect downstream application scenarios are
essential for the streamlined adoption of new research in tabular machine
learning (ML). In this work, we examine existing tabular benchmarks and find
two common characteristics of industry-grade tabular data that are
underrepresented in the datasets available to the academic community. First,
tabular data often changes over time in real-world deployment scenarios. This
impacts model performance and requires time-based train and test splits for
correct model evaluation. Yet, existing academic tabular datasets often lack
timestamp metadata to enable such evaluation. Second, a considerable portion of
datasets in production settings stem from extensive data acquisition and
feature engineering pipelines. For each specific dataset, this can have a
different impact on the absolute and relative number of predictive,
uninformative, and correlated features, which in turn can affect model
selection. To fill the aforementioned gaps in academic benchmarks, we introduce
TabReD -- a collection of eight industry-grade tabular datasets covering a wide
range of domains from finance to food delivery services. We assess a large
number of tabular ML models in the feature-rich, temporally-evolving data
setting facilitated by TabReD. We demonstrate that evaluation on time-based
data splits leads to different methods ranking, compared to evaluation on
random splits more common in academic benchmarks. Furthermore, on the TabReD
datasets, MLP-like architectures and GBDT show the best results, while more
sophisticated DL models are yet to prove their effectiveness.

本文研究了标准机器学习中现成的表格式基准数据集存在的问题，并引入了 TabReD 数据集以填补学术基准数据集的不足之处，以此评估不同 ML 模型在时间演化的数据环境中的表现。结果表明，基于时间的数据划分与学术基准数据集中的随机划分会导致不同的方法排名，MLP 类的架构和 GBDT 模型在 TabReD 数据集上展现出最佳结果，而更复杂的深度学习模型还需进一步证明其有效性。

TabReD：一个野外表格机器学习基准

TabReD: A Benchmark of Tabular Machine Learning in-the-Wild

Recent research has explored how Language Models (LMs) can be used for
feature representation and prediction in tabular machine learning tasks. This
involves employing text serialization and supervised fine-tuning (SFT)
techniques. Despite the simplicity of these techniques, significant gaps remain
in our understanding of the applicability and reliability of LMs in this
context. Our study assesses how emerging LM technologies compare with
traditional paradigms in tabular machine learning and evaluates the feasibility
of adopting similar approaches with these advanced technologies. At the data
level, we investigate various methods of data representation and curation of
serialized tabular data, exploring their impact on prediction performance. At
the classification level, we examine whether text serialization combined with
LMs enhances performance on tabular datasets (e.g. class imbalance,
distribution shift, biases, and high dimensionality), and assess whether this
method represents a state-of-the-art (SOTA) approach for addressing tabular
machine learning challenges. Our findings reveal current pre-trained models
should not replace conventional approaches.

最近的研究探讨了语言模型如何在表格机器学习任务中用作特征表示和预测的方法，通过使用文本串行化和有监督的微调技术，我们评估了这些新兴的语言模型技术与传统模式在表格机器学习中的适用性和可靠性，并调查了数据表示和整理方式对预测性能的影响。结果显示，当前的预训练模型不应取代传统方法。