Training reliable deep learning models which avoid making overconfident but
incorrect predictions is a longstanding challenge. This challenge is further
exacerbated when learning has to be differentially private: protection provided
to sensitive data comes at the price of injecting additional randomness into
the learning process. In this work, we conduct a thorough empirical
investigation of selective classifiers -- that can abstain when they are unsure
-- under a differential privacy constraint. We find that several popular
selective prediction approaches are ineffective in a differentially private
setting as they increase the risk of privacy leakage. At the same time, we
identify that a recent approach that only uses checkpoints produced by an
off-the-shelf private learning algorithm stands out as particularly suitable
under DP. Further, we show that differential privacy does not just harm utility
but also degrades selective classification performance. To analyze this effect
across privacy levels, we propose a novel evaluation mechanism which isolate
selective prediction performance across model utility levels. Our experimental
results show that recovering the performance level attainable by non-private
models is possible but comes at a considerable coverage cost as the privacy
budget decreases.

通过研究选择性分类器在差分隐私约束下的效果，探讨深度学习模型的可靠性及隐私泄漏问题，发现最近的一种基于现成的深度学习模型生成检查点的方法在差分隐私下更为合适，使用差分隐私不仅会降低模型的效能，而且在隐私预算降低时需要付出相当大的覆盖成本。