Large language models (LLMs) have demonstrated their prowess in generating
synthetic text and images; however, their potential for generating tabular data
-- arguably the most common data type in business and scientific applications
-- is largely underexplored. This paper demonstrates that LLMs, used as-is, or
after traditional fine-tuning, are severely inadequate as synthetic table
generators. Due to the autoregressive nature of LLMs, fine-tuning with random
order permutation runs counter to the importance of modeling functional
dependencies, and renders LLMs unable to model conditional mixtures of
distributions (key to capturing real world constraints). We showcase how LLMs
can be made to overcome some of these deficiencies by making them
permutation-aware.

大语言模型在生成合成文本和图像方面表现出色；然而，在生成表格数据方面，它们的潜力相对较少。本文展示了使用大语言模型，无论是原样使用还是传统的微调后使用，作为合成表格生成器是明显不足的。通过使用排列注意力，我们展示了如何使大语言模型克服其中的一些缺陷。