With the growing demand for synthetic data to address contemporary issues in
machine learning, such as data scarcity, data fairness, and data privacy,
having robust tools for assessing the utility and potential privacy risks of
such data becomes crucial. SynthEval, a novel open-source evaluation framework
distinguishes itself from existing tools by treating categorical and numerical
attributes with equal care, without assuming any special kind of preprocessing
steps. This~makes it applicable to virtually any synthetic dataset of tabular
records. Our tool leverages statistical and machine learning techniques to
comprehensively evaluate synthetic data fidelity and privacy-preserving
integrity. SynthEval integrates a wide selection of metrics that can be used
independently or in highly customisable benchmark configurations, and can
easily be extended with additional metrics. In this paper, we describe
SynthEval and illustrate its versatility with examples. The framework
facilitates better benchmarking and more consistent comparisons of model
capabilities.

合成数据评估框架 SynthEval 旨在解决机器学习中的数据稀缺性、数据公平性和数据隐私性问题，并通过统计和机器学习技术全面评估数据的准确性和隐私保护完整性。

SynthEval：一个用于详细评估表格化合数据的效用和隐私的框架

SynthEval: A Framework for Detailed Utility and Privacy Evaluation of  Tabular Synthetic Data

High-throughput drug screening -- using cell imaging or gene expression
measurements as readouts of drug effect -- is a critical tool in biotechnology
to assess and understand the relationship between the chemical structure and
biological activity of a drug. Since large-scale screens have to be divided
into multiple experiments, a key difficulty is dealing with batch effects,
which can introduce systematic errors and non-biological associations in the
data. We propose InfoCORE, an Information maximization approach for COnfounder
REmoval, to effectively deal with batch effects and obtain refined molecular
representations. InfoCORE establishes a variational lower bound on the
conditional mutual information of the latent representations given a batch
identifier. It adaptively reweighs samples to equalize their implied batch
distribution. Extensive experiments on drug screening data reveal InfoCORE's
superior performance in a multitude of tasks including molecular property
prediction and molecule-phenotype retrieval. Additionally, we show results for
how InfoCORE offers a versatile framework and resolves general distribution
shifts and issues of data fairness by minimizing correlation with spurious
features or removing sensitive attributes. The code is available at
this https URL

高通量药物筛选是生物技术中评估药物化学结构与生物活性关系的关键工具。本文提出了一种信息最大化方法 InfoCORE 来有效处理批次效应，并获得精确的分子表征。InfoCORE 通过建立给定批次标识符的潜在表征的条件互信息的变分下界，自适应地重新加权样本以均衡其隐含批次分布。大量的药物筛选数据实验证明 InfoCORE 在包括分子性质预测和分子 - 表型检索等多项任务中具有卓越性能。此外，我们展示了 InfoCORE 如何通过最小化与虚假特征的相关性或移除敏感属性来提供一个多用途的框架，解决了数据分布偏移和数据公平性的问题。