Machine learning (ML) models trained by differentially private stochastic
gradient descent (DP-SGD) have much lower utility than the non-private ones. To
mitigate this degradation, we propose a DP Laplacian smoothing SGD (DP-LSSGD)
to train ML models with differential privacy (DP) guarantees. At the core of
DP-LSSGD is the Laplacian smoothing, which smooths out the Gaussian noise used
in the Gaussian mechanism. Under the same amount of noise used in the Gaussian
mechanism, DP-LSSGD attains the same DP guarantee, but in practice, DP-LSSGD
makes training both convex and nonconvex ML models more stable and enables the
trained models to generalize better. The proposed algorithm is simple to
implement and the extra computational complexity and memory overhead compared
with DP-SGD are negligible. DP-LSSGD is applicable to train a large variety of
ML models, including DNNs. The code is available at
https://github.com/BaoWangMath/DP-LSSGD.

本研究提出 DP-LSSGD，一种基于差分隐私的 SGD 算法，通过 Laplacian smoothing 减少噪音并保证同样的隐私保证，让训练凸和非凸机器学习模型更加稳定，同时使得模型泛化更好。