Due to massive amounts of data distributed across multiple locations, distributed machine learning has attracted a lot of research interests. Alternating Direction Method of Multipliers (ADMM) is a powerful method of designing distributed machine learning algorithm, whereby each agent computes over local datasets and exchanges computation results with its neighbor agents in an iterative procedure. There exists significant privacy leakage during this iterative process if the local data is sensitive. In this paper, we propose a differentially private ADMM algorithm (P-ADMM) to provide dynamic zero-concentrated differential privacy (dynamic zCDP), by inserting Gaussian noise with linearly decaying variance. We prove that P-ADMM has the same convergence rate compared to the non-private counterpart, i.e., $\mathcal{O}(1/K)$ with $K$ being the number of iterations and linear convergence for general convex and strongly convex problems while providing differentially private guarantee. Moreover, through our experiments performed on real-world datasets, we empirically show that P-ADMM has the best-known performance among the existing differentially private ADMM based algorithms.

本文提出了使用差分隐私的Alternating Direction Method of Multipliers（P-ADMM）算法，使得在分布式机器学习中每个本地数据集与邻近代理交换计算结果的迭代过程中保持数据隐私不泄露，并证明这一算法与非私有算法有着相同的收敛率, 实验结果表明P-ADMM在现有的基于ADMM的差分隐私算法中表现最佳。

分布式医疗机器学习的差分隐私ADMM