Data synthesis has been advocated as an important approach for utilizing data
while protecting data privacy. A large number of tabular data synthesis
algorithms (which we call synthesizers) have been proposed. Some synthesizers
satisfy Differential Privacy, while others aim to provide privacy in a
heuristic fashion. A comprehensive understanding of the strengths and
weaknesses of these synthesizers remains elusive due to lacking principled
evaluation metrics and missing head-to-head comparisons of newly developed
synthesizers that take advantage of diffusion models and large language models
with state-of-the-art marginal-based synthesizers.
In this paper, we present a principled and systematic evaluation framework
for assessing tabular data synthesis algorithms. Specifically, we examine and
critique existing evaluation metrics, and introduce a set of new metrics in
terms of fidelity, privacy, and utility to address their limitations. Based on
the proposed metrics, we also devise a unified objective for tuning, which can
consistently improve the quality of synthetic data for all methods. We
conducted extensive evaluations of 8 different types of synthesizers on 12
datasets and identified some interesting findings, which offer new directions
for privacy-preserving data synthesis.

利用扩散模型和大型语言模型的新型综合器进行对比评估，提出了一套评估指标，旨在解决现有评估指标的局限性，并为调优提供了普遍目标，以保护数据隐私和提高合成数据的质量。