In this paper, we propose a novel Heterogeneous Gaussian Mechanism (HGM) to preserve differential privacy in deep neural networks, with provable robustness against adversarial examples. We first relax the constraint of the privacy budget in the traditional Gaussian Mechanism from (0, 1] to (0, \infty), with a new bound of the noise scale to preserve differential privacy. The noise in our mechanism can be arbitrarily redistributed, offering a distinctive ability to address the trade-off between model utility and privacy loss. To derive provable robustness, our HGM is applied to inject Gaussian noise into the first hidden layer. Then, a tighter robustness bound is proposed. Theoretical analysis and thorough evaluations show that our mechanism notably improves the robustness of differentially private deep neural networks, compared with baseline approaches, under a variety of model attacks.

该研究提出了一种新型的异构高斯机制（HGM），用于保护深度神经网络的差分隐私，并具有针对对抗性攻击的可证明强鲁棒性。HGM可以提供灵活的噪声分布来平衡模型效用和隐私损失，并在第一个隐藏层注入高斯噪声以获得强鲁棒性保护。该研究理论分析和实验验证显示，相比基准方法，HGM机制显着提高了不同ially private深度神经网络在各种模型攻击下的鲁棒性。

异构高斯机制：在具有可证健壮性的深度学习中保持差分隐私