In privacy-preserving machine learning, differentially private stochastic
gradient descent (DP-SGD) performs worse than SGD due to per-sample gradient
clipping and noise addition. A recent focus in private learning research is
improving the performance of DP-SGD on private data by incorporating priors
that are learned on real-world public data. In this work, we explore how we can
improve the privacy-utility tradeoff of DP-SGD by learning priors from images
generated by random processes and transferring these priors to private data. We
propose DP-RandP, a three-phase approach. We attain new state-of-the-art
accuracy when training from scratch on CIFAR10, CIFAR100, and MedMNIST for a
range of privacy budgets $\varepsilon \in [1, 8]$. In particular, we improve
the previous best reported accuracy on CIFAR10 from $60.6 \%$ to $72.3 \%$ for
$\varepsilon=1$. Our code is available at
this https URL

本研究提出了 DP-RandP 方法，并从随机过程生成的图像中学习先验知识，并将这些先验知识传递给私有数据，从而提高了差分隐私随机梯度下降 DP-SGD 的隐私效用权衡的性能，并在 CIFAR10，CIFAR100 和 MedMNIST 数据集上实现了新的最佳准确度。