Federated learning (FL) has been recognized as a rapidly growing research
area, where the model is trained over massively distributed clients under the
orchestration of a parameter server (PS) without sharing clients' data. This
paper delves into a class of federated problems characterized by non-convex and
non-smooth loss functions, that are prevalent in FL applications but
challenging to handle due to their intricate non-convexity and non-smoothness
nature and the conflicting requirements on communication efficiency and privacy
protection. In this paper, we propose a novel federated primal-dual algorithm
with bidirectional model sparsification tailored for non-convex and non-smooth
FL problems, and differential privacy is applied for strong privacy guarantee.
Its unique insightful properties and some privacy and convergence analyses are
also presented for the FL algorithm design guidelines. Extensive experiments on
real-world data are conducted to demonstrate the effectiveness of the proposed
algorithm and much superior performance than some state-of-the-art FL
algorithms, together with the validation of all the analytical results and
properties.

提出了一种用于非凸和非平滑联邦学习问题的新型原始 - 对偶算法，结合双向模型稀疏化和差分隐私，以保证强隐私，通过真实数据上的大量实验验证了该算法的有效性和比某些最先进的联邦学习算法更出色的性能，并验证了所有分析结果和特性。

保护隐私的联邦原始 - 对偶学习用于模型稀疏化的非凸和非平滑问题

Privacy-preserving Federated Primal-dual Learning for Non-convex and  Non-smooth Problems with Model Sparsification

We establish generic uniform convergence guarantees for Gaussian data in
terms of the Rademacher complexity of the hypothesis class and the Lipschitz
constant of the square root of the scalar loss function. We show how these
guarantees substantially generalize previous results based on smoothness
(Lipschitz constant of the derivative), and allow us to handle the broader
class of square-root-Lipschitz losses, which includes also non-smooth loss
functions appropriate for studying phase retrieval and ReLU regression, as well
as rederive and better understand "optimistic rate" and interpolation learning
guarantees.

本文研究基于 Rademacher 复杂度和平方根标量损失函数的 Lipschitz 常数，以高斯数据的泛化一致性保证为例，并处理了广泛的平方根 Lipschitz 损失函数的类别，包括适用于相位恢复和 ReLU 回归的非平滑损失函数，从而重新推导和更好地理解 “乐观率” 和插值学习保证。