Recent research has explored how Language Models (LMs) can be used for
feature representation and prediction in tabular machine learning tasks. This
involves employing text serialization and supervised fine-tuning (SFT)
techniques. Despite the simplicity of these techniques, significant gaps remain
in our understanding of the applicability and reliability of LMs in this
context. Our study assesses how emerging LM technologies compare with
traditional paradigms in tabular machine learning and evaluates the feasibility
of adopting similar approaches with these advanced technologies. At the data
level, we investigate various methods of data representation and curation of
serialized tabular data, exploring their impact on prediction performance. At
the classification level, we examine whether text serialization combined with
LMs enhances performance on tabular datasets (e.g. class imbalance,
distribution shift, biases, and high dimensionality), and assess whether this
method represents a state-of-the-art (SOTA) approach for addressing tabular
machine learning challenges. Our findings reveal current pre-trained models
should not replace conventional approaches.

最近的研究探讨了语言模型如何在表格机器学习任务中用作特征表示和预测的方法，通过使用文本串行化和有监督的微调技术，我们评估了这些新兴的语言模型技术与传统模式在表格机器学习中的适用性和可靠性，并调查了数据表示和整理方式对预测性能的影响。结果显示，当前的预训练模型不应取代传统方法。