In recent years, privacy-preserving machine learning algorithms have attracted increasing attention because of their important applications in many scientific fields. However, in the literature, most privacy-preserving algorithms demand learning objectives to be strongly convex and Lipschitz smooth, which thus cannot cover a wide class of robust loss functions (e.g., quantile/least absolute loss). In this work, we aim to develop a fast privacy-preserving learning solution for a sparse robust regression problem. Our learning loss consists of a robust least absolute loss and an $\ell_1$ sparse penalty term. To fast solve the non-smooth loss under a given privacy budget, we develop a Fast Robust And Privacy-Preserving Estimation (FRAPPE) algorithm for least absolute deviation regression. Our algorithm achieves a fast estimation by reformulating the sparse LAD problem as a penalized least square estimation problem and adopts a three-stage noise injection to guarantee the $(\epsilon,\delta)$-differential privacy. We show that our algorithm can achieve better privacy and statistical accuracy trade-off compared with the state-of-the-art privacy-preserving regression algorithms. In the end, we conduct experiments to verify the efficiency of our proposed FRAPPE algorithm.

本文提出了一种用于稀疏鲁棒回归问题的快速隐私保护学习解决方案，其中学习损失包括鲁棒最小绝对值损失和$l_1$稀疏惩罚项。为了快速解决给定隐私预算下的非光滑损失，我们开发了一种名为FRAPPE的算法，通过将稀疏LAD问题重新表述为带惩罚的最小二乘估计问题，并采用三阶段噪声注入来保证$(\epsilon,\delta)$-差分隐私。实验证明，与现有隐私保护回归算法相比，我们的算法在隐私和统计准确性的权衡上取得了更好的性能。

高效稀疏最小绝对偏差回归与差分隐私