Recent breakthroughs in large language modeling have facilitated rigorous
exploration of their application in diverse tasks related to tabular data
modeling, such as prediction, tabular data synthesis, question answering, and
table understanding. Each task presents unique challenges and opportunities.
However, there is currently a lack of comprehensive review that summarizes and
compares the key techniques, metrics, datasets, models, and optimization
approaches in this research domain. This survey aims to address this gap by
consolidating recent progress in these areas, offering a thorough survey and
taxonomy of the datasets, metrics, and methodologies utilized. It identifies
strengths, limitations, unexplored territories, and gaps in the existing
literature, while providing some insights for future research directions in
this vital and rapidly evolving field. It also provides relevant code and
datasets references. Through this comprehensive review, we hope to provide
interested readers with pertinent references and insightful perspectives,
empowering them with the necessary tools and knowledge to effectively navigate
and address the prevailing challenges in the field.

综述中介绍了大规模语言建模在表格数据建模相关任务中的应用，包括预测、表格数据合成、问答和表格理解。通过归纳最新进展和总结数据集、度量标准和方法论，本综述发现了现有领域文献的优势、局限性、未开发的领域和研究间隙，并提供了相关代码和数据集的参考。期望以此为读者提供有关参考和深入见解，为他们在这个重要且快速进展的领域中有效应对挑战提供必备的工具和知识。

大型语言模型在表格数据上的应用 -- 综述

Large Language Models on Tabular Data -- A Survey

Until recently, the question of the effective inductive bias of deep models
on tabular data has remained unanswered. This paper investigates the hypothesis
that arithmetic feature interaction is necessary for deep tabular learning. To
test this point, we create a synthetic tabular dataset with a mild feature
interaction assumption and examine a modified transformer architecture enabling
arithmetical feature interactions, referred to as AMFormer. Results show that
AMFormer outperforms strong counterparts in fine-grained tabular data modeling,
data efficiency in training, and generalization. This is attributed to its
parallel additive and multiplicative attention operators and prompt-based
optimization, which facilitate the separation of tabular samples in an extended
space with arithmetically-engineered features. Our extensive experiments on
real-world data also validate the consistent effectiveness, efficiency, and
rationale of AMFormer, suggesting it has established a strong inductive bias
for deep learning on tabular data. Code is available at
this https URL

本文研究了深度模型在表格数据上的有效归纳偏差问题，提出了具有算术特征交互的 AMFormer 变形器结构，并在合成和真实数据上进行了广泛实验，证明了 AMFormer 在细粒度表格数据建模、训练数据效率和泛化能力方面的优势，从而表明它在表格数据深度学习中建立了强有力的归纳偏差。