To prevent unintentional data leakage, research community has resorted to data generators that can produce differentially private data for model training. However, for the sake of the data privacy, existing solutions suffer from either expensive training cost or poor generalization performance. Therefore, we raise the question whether training efficiency and privacy can be achieved simultaneously. In this work, we for the first time identify that dataset condensation (DC) which is originally designed for improving training efficiency is also a better solution to replace the traditional data generators for private data generation, thus providing privacy for free. To demonstrate the privacy benefit of DC, we build a connection between DC and differential privacy, and theoretically prove on linear feature extractors (and then extended to non-linear feature extractors) that the existence of one sample has limited impact ($O(m/n)$) on the parameter distribution of networks trained on $m$ samples synthesized from $n (n \gg m)$ raw samples by DC. We also empirically validate the visual privacy and membership privacy of DC-synthesized data by launching both the loss-based and the state-of-the-art likelihood-based membership inference attacks. We envision this work as a milestone for data-efficient and privacy-preserving machine learning.

本文首次发现，原本用于提高训练效率的数据集压缩技术(dataset condensation, DC)，在数据隐私保护方面也是替代传统数据生成器的更好解决方案。我们提出了在DC中提供隐私保护，并理论上证明DC合成的$m$个样本在参数分布上对网络的影响有限($O(m/n)$)，同时通过试验证明DCsynthesized数据的可视隐私和成员隐私。本文的创新之处在于：通过数据压缩实现了数据高效性与隐私保护的双赢。

免费提供隐私：数据集压缩如何帮助隐私？