This paper proposes a locally differentially private federated learning
algorithm for strongly convex but possibly nonsmooth problems that protects the
gradients of each worker against an honest but curious server. The proposed
algorithm adds artificial noise to the shared information to ensure privacy and
dynamically allocates the time-varying noise variance to minimize an upper
bound of the optimization error subject to a predefined privacy budget
constraint. This allows for an arbitrarily large but finite number of
iterations to achieve both privacy protection and utility up to a neighborhood
of the optimal solution, removing the need for tuning the number of iterations.
Numerical results show the superiority of the proposed algorithm over
state-of-the-art methods.

本文提出了一种局部差分隐私联邦学习算法，用于强凸但可能非光滑问题，以保护每个参与者的梯度免受诚实但好奇的服务器的侵犯，并通过给共享信息添加人工噪声来保证隐私，并动态分配时变噪声方差以最小化一个预定义的隐私预算约束下的优化误差上界，从而可以在无需调整迭代次数的情况下，在可接近最优解的周围实现隐私保护和效用，数值结果表明该算法优于现有的方法。

基于局部差异隐私联邦学习的动态隐私分配与复合目标

Dynamic Privacy Allocation for Locally Differentially Private Federated  Learning with Composite Objectives

Stochastic optimization has found wide applications in minimizing objective
functions in machine learning, which motivates a lot of theoretical studies to
understand its practical success. Most of existing studies focus on the
convergence of optimization errors, while the generalization analysis of
stochastic optimization is much lagging behind. This is especially the case for
nonconvex and nonsmooth problems often encountered in practice. In this paper,
we initialize a systematic stability and generalization analysis of stochastic
optimization on nonconvex and nonsmooth problems. We introduce novel
algorithmic stability measures and establish their quantitative connection on
the gap between population gradients and empirical gradients, which is then
further extended to study the gap between the Moreau envelope of the empirical
risk and that of the population risk. To our knowledge, these quantitative
connection between stability and generalization in terms of either gradients or
Moreau envelopes have not been studied in the literature. We introduce a class
of sampling-determined algorithms, for which we develop bounds for three
stability measures. Finally, we apply these discussions to derive error bounds
for stochastic gradient descent and its adaptive variant, where we show how to
achieve an implicit regularization by tuning the step sizes and the number of
iterations.

本文针对非凸非光滑问题提出新的算法稳定性度量方法，同时建立它们与梯度之间的量化关系，并使用采样确定算法导出了随机梯度下降算法和其自适应变种的误差界。