Training generative models to produce synthetic data is meant to provide a
privacy-friendly approach to data release. However, we get robust guarantees
only when models are trained to satisfy Differential Privacy (DP). Alas, this
is not the standard in industry as many companies use ad-hoc strategies to
empirically evaluate privacy based on the statistical similarity between
synthetic and real data. In this paper, we review the privacy metrics offered
by leading companies in this space and shed light on a few critical flaws in
reasoning about privacy entirely via empirical evaluations. We analyze the
undesirable properties of the most popular metrics and filters and demonstrate
their unreliability and inconsistency through counter-examples. We then present
a reconstruction attack, ReconSyn, which successfully recovers (i.e., leaks all
attributes of) at least 78% of the low-density train records (or outliers) with
only black-box access to a single fitted generative model and the privacy
metrics. Finally, we show that applying DP only to the model or using
low-utility generators does not mitigate ReconSyn as the privacy leakage
predominantly comes from the metrics. Overall, our work serves as a warning to
practitioners not to deviate from established privacy-preserving mechanisms.

通过评估数据的统计相似性来衡量隐私是不可靠和不一致的，而生成模型只应以差分隐私为标准以保护数据隐私。本文针对领先公司提供的隐私度量指标进行分析，揭示了一些关键的推理缺陷，并提出了成功恢复生成模型中绝大部分用于训练的异常记录的重构攻击 ReconSyn。研究表明，仅将差分隐私应用于模型或使用低效的生成器不能减轻 ReconSyn 对隐私的泄露风险，因为泄露主要来自于度量指标。总体而言，我们的工作提醒从业人员不要偏离既定的隐私保护机制。

相似性为基础的隐私测量的不足：针对 “真正匿名合成数据” 的重构攻击

On the Inadequacy of Similarity-based Privacy Metrics: Reconstruction  Attacks against "Truly Anonymous Synthetic Data''

The goal of privacy metrics is to measure the degree of privacy enjoyed by
users in a system and the amount of protection offered by privacy-enhancing
technologies. In this way, privacy metrics contribute to improving user privacy
in the digital world. The diversity and complexity of privacy metrics in the
literature makes an informed choice of metrics challenging. As a result,
instead of using existing metrics, new metrics are proposed frequently, and
privacy studies are often incomparable. In this survey we alleviate these
problems by structuring the landscape of privacy metrics. To this end, we
explain and discuss a selection of over eighty privacy metrics and introduce
categorizations based on the aspect of privacy they measure, their required
inputs, and the type of data that needs protection. In addition, we present a
method on how to choose privacy metrics based on nine questions that help
identify the right privacy metrics for a given scenario, and highlight topics
where additional work on privacy metrics is needed. Our survey spans multiple
privacy domains and can be understood as a general framework for privacy
measurement.

本文回顾了 80 多个隐私度量标准，介绍了衡量隐私度量标准的方面、必要输入、需要保护的数据类型，并提出了基于问题的选择隐私指标的方法，以及需要进一步研究隐私度量标准的主题。该研究跨越多个隐私领域，是一个可以理解为隐私测量的一般框架。

技术隐私度量：系统性调查

Technical Privacy Metrics: a Systematic Survey

We propose a general statistical inference framework to capture the privacy
threat incurred by a user that releases data to a passive but curious
adversary, given utility constraints. We show that applying this general
framework to the setting where the adversary uses the self-information cost
function naturally leads to a non-asymptotic information-theoretic approach for
characterizing the best achievable privacy subject to utility constraints.
Based on these results we introduce two privacy metrics, namely average
information leakage and maximum information leakage. We prove that under both
metrics the resulting design problem of finding the optimal mapping from the
user's data to a privacy-preserving output can be cast as a modified
rate-distortion problem which, in turn, can be formulated as a convex program.
Finally, we compare our framework with differential privacy.

提出一种用于捕获用户向被动好奇敌手发布数据时产生的隐私威胁并在效用约束下加以限制的一般性统计推断框架，其中应用自信息成本函数于非渐进信息论方法用于表征最佳可实现的隐私，基于这些结果引入平均信息泄漏和最大信息泄漏两种隐私度量，证明在两种指标下，将用户数据映射到隐私保护输出的最优映射可被描述为修正的率失真问题并可设计成凸编程，最后与差分隐私进行比较。