Federated Learning (FL) is a distributed machine learning approach to learn
models on decentralized heterogeneous data, without the need for clients to
share their data. Many existing FL approaches assume that all clients have
equal importance and construct a global objective based on all clients. We
consider a version of FL we call Prioritized FL, where the goal is to learn a
weighted mean objective of a subset of clients, designated as priority clients.
An important question arises: How do we choose and incentivize well aligned non
priority clients to participate in the federation, while discarding misaligned
clients? We present FedALIGN (Federated Adaptive Learning with Inclusion of
Global Needs) to address this challenge. The algorithm employs a matching
strategy that chooses non priority clients based on how similar the models loss
is on their data compared to the global data, thereby ensuring the use of non
priority client gradients only when it is beneficial for priority clients. This
approach ensures mutual benefits as non priority clients are motivated to join
when the model performs satisfactorily on their data, and priority clients can
utilize their updates and computational resources when their goals align. We
present a convergence analysis that quantifies the trade off between client
selection and speed of convergence. Our algorithm shows faster convergence and
higher test accuracy than baselines for various synthetic and benchmark
datasets.

FedALIGN 是一种选择参与联邦学习的非优先客户的匹配策略，基于他们的模型损失与全局数据的相似程度，以确保只有在对优先客户有益时才使用非优先客户的梯度，从而实现非优先客户的加入动机和优先客户的资源利用，且在各种合成和基准数据集上展示出更快的收敛速度和更高的测试准确性。

利用自由客户在联邦学习中进行模型增强

Utilizing Free Clients in Federated Learning for Focused Model  Enhancement

Federated learning (FL) is a promising strategy for performing
privacy-preserving, distributed learning with a network of clients (i.e., edge
devices). However, the data distribution among clients is often non-IID in
nature, making efficient optimization difficult. To alleviate this issue, many
FL algorithms focus on mitigating the effects of data heterogeneity across
clients by introducing a variety of proximal terms, some incurring considerable
compute and/or memory overheads, to restrain local updates with respect to the
global model. Instead, we consider rethinking solutions to data heterogeneity
in FL with a focus on local learning generality rather than proximal
restriction. To this end, we first present a systematic study informed by
second-order indicators to better understand algorithm effectiveness in FL.
Interestingly, we find that standard regularization methods are surprisingly
strong performers in mitigating data heterogeneity effects. Based on our
findings, we further propose a simple and effective method, FedAlign, to
overcome data heterogeneity and the pitfalls of previous methods. FedAlign
achieves competitive accuracy with state-of-the-art FL methods across a variety
of settings while minimizing computation and memory overhead. Code is available
at this https URL

本文提出了一种叫做 FedAlign 的基于局部学习广泛性而非接近约束的数据异构性联邦学习解决方案，其可在不引入大量计算和内存开销的情况下实现与最先进的 FL 方法相当的精度。