We consider the privacy-preserving machine learning (ML) setting where the
trained model must satisfy differential privacy (DP) with respect to the labels
of the training examples. We propose two novel approaches based on,
respectively, the Laplace mechanism and the PATE framework, and demonstrate
their effectiveness on standard benchmarks.
While recent work by Ghazi et al. proposed Label DP schemes based on a
randomized response mechanism, we argue that additive Laplace noise coupled
with Bayesian inference (ALIBI) is a better fit for typical ML tasks. Moreover,
we show how to achieve very strong privacy levels in some regimes, with our
adaptation of the PATE framework that builds on recent advances in
semi-supervised learning.
We complement theoretical analysis of our algorithms' privacy guarantees with
empirical evaluation of their memorization properties. Our evaluation suggests
that comparing different algorithms according to their provable DP guarantees
can be misleading and favor a less private algorithm with a tighter analysis.
Code for implementation of algorithms and memorization attacks is available
from this https URL

本研究提出了两种新的隐私保护机制，分别基于 Laplace 机制和 PATE 框架，用标准基准测试表明其有效性；此外，本文也表明其算法隐私保护保障的理论分析与实证评估相结合，才能更好地评价其隐私保护性能。