In this paper, we consider efficient differentially private empirical risk
minimization from the viewpoint of optimization algorithms. For strongly convex
and smooth objectives, we prove that gradient descent with output perturbation
not only achieves nearly optimal utility, but also significantly improves the
running time of previous state-of-the-art private optimization algorithms, for
both $\epsilon$-DP and $(\epsilon, \delta)$-DP. For non-convex but smooth
objectives, we propose an RRPSGD (Random Round Private Stochastic Gradient
Descent) algorithm, which provably converges to a stationary point with privacy
guarantee. Besides the expected utility bounds, we also provide guarantees in
high probability form. Experiments demonstrate that our algorithm consistently
outperforms existing method in both utility and running time.

本文综述了不同凸且光滑目标函数的梯度下降算法与输出扰动方法，不仅能达到近乎最优的效用，而且也提高了以前的隐私优化算法的运行时间。同时，对于非凸光滑目标函数，还提出了 RRPSGD 算法，并附上了隐私保障的收敛性证明。实验表明，我们的算法在效益和运行时间方面始终优于现有方法。