Federated Learning (FL) is a decentralized machine learning (ML) approach
that keeps data localized and often incorporates Differential Privacy (DP) to
enhance privacy guarantees. Similar to previous work on DP in ML, we observed
that differentially private federated learning (DPFL) introduces performance
disparities, particularly affecting minority groups. Recent work has attempted
to address performance fairness in vanilla FL through clustering, but this
method remains sensitive and prone to errors, which are further exacerbated by
the DP noise in DPFL. To fill this gap, in this paper, we propose a novel
clustered DPFL algorithm designed to effectively identify clients' clusters in
highly heterogeneous settings while maintaining high accuracy with DP
guarantees. To this end, we propose to cluster clients based on both their
model updates and training loss values. Our proposed approach also addresses
the server's uncertainties in clustering clients' model updates by employing
larger batch sizes along with Gaussian Mixture Model (GMM) to alleviate the
impact of noise and potential clustering errors, especially in
privacy-sensitive scenarios. We provide theoretical analysis of the
effectiveness of our proposed approach. We also extensively evaluate our
approach across diverse data distributions and privacy budgets and show its
effectiveness in mitigating the disparate impact of DP in FL settings with a
small computational cost.

提出了一种新颖的聚类差分私有化联邦学习算法，通过基于模型更新和训练损失值对客户进行聚类，以应对联邦学习中的不平等性和隐私保护问题。

通过健壮聚类减轻联邦学习中差异隐私的不平等影响

Mitigating Disparate Impact of Differential Privacy in Federated  Learning through Robust Clustering

Differentially Private Federated Learning (DPFL) is an emerging field with
many applications. Gradient averaging based DPFL methods require costly
communication rounds and hardly work with large-capacity models, due to the
explicit dimension dependence in its added noise. In this work, inspired by
knowledge transfer non-federated privacy learning from Papernot et al.(2017;
2018), we design two new DPFL schemes, by voting among the data labels returned
from each local model, instead of averaging the gradients, which avoids the
dimension dependence and significantly reduces the communication cost.
Theoretically, by applying secure multi-party computation, we could
exponentially amplify the (data-dependent) privacy guarantees when the margin
of the voting scores are large. Extensive experiments show that our approaches
significantly improve the privacy-utility trade-off over the state-of-the-arts
in DPFL.

本研究探讨在差分隐私联邦学习的领域中，通过知识迁移和基于数据标签投票的方法替代梯度平均来减少通信成本和避免维度依赖性，从而提高了差分隐私保证与模型效用的平衡。