The performance of Federated Learning (FL) hinges on the effectiveness of
utilizing knowledge from distributed datasets. Traditional FL methods adopt an
aggregate-then-adapt framework, where clients update local models based on a
global model aggregated by the server from the previous training round. This
process can cause client drift, especially with significant cross-client data
heterogeneity, impacting model performance and convergence of the FL algorithm.
To address these challenges, we introduce FedAF, a novel aggregation-free FL
algorithm. In this framework, clients collaboratively learn condensed data by
leveraging peer knowledge, the server subsequently trains the global model
using the condensed data and soft labels received from the clients. FedAF
inherently avoids the issue of client drift, enhances the quality of condensed
data amid notable data heterogeneity, and improves the global model
performance. Extensive numerical studies on several popular benchmark datasets
show FedAF surpasses various state-of-the-art FL algorithms in handling
label-skew and feature-skew data heterogeneity, leading to superior global
model accuracy and faster convergence.

FedAF 是一种不需要数据聚合的联邦学习算法，通过协作学习来减少数据异质性导致的客户端漂移问题，提高全局模型准确性和收敛速度

解决数据异构的无聚合联邦学习

An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

While federated learning (FL) systems often utilize quantization to battle
communication and computational bottlenecks, they have heretofore been limited
to deploying fixed-precision quantization schemes. Meanwhile, the concept of
mixed-precision quantization (MPQ), where different layers of a deep learning
model are assigned varying bit-width, remains unexplored in the FL settings. We
present a novel FL algorithm, FedMPQ, which introduces mixed-precision
quantization to resource-heterogeneous FL systems. Specifically, local models,
quantized so as to satisfy bit-width constraint, are trained by optimizing an
objective function that includes a regularization term which promotes reduction
of precision in some of the layers without significant performance degradation.
The server collects local model updates, de-quantizes them into full-precision
models, and then aggregates them into a global model. To initialize the next
round of local training, the server relies on the information learned in the
previous training round to customize bit-width assignments of the models
delivered to different clients. In extensive benchmarking experiments on
several model architectures and different datasets in both iid and non-iid
settings, FedMPQ outperformed the baseline FL schemes that utilize
fixed-precision quantization while incurring only a minor computational
overhead on the participating devices.

引入混合精度量化方法到异构资源联邦学习系统中以解决通信和计算瓶颈问题，并在多个模型架构和数据集上进行了广泛的基准性实验验证其优于固定精度量化的性能。

资源受限异构设备上的联邦学习的混合精度量化

Mixed-Precision Quantization for Federated Learning on  Resource-Constrained Heterogeneous Devices

Federated learning (FL) is a distributed learning paradigm that enables
multiple clients to learn a powerful global model by aggregating local
training. However, the performance of the global model is often hampered by
non-i.i.d. distribution among the clients, requiring extensive efforts to
mitigate inter-client data heterogeneity. Going beyond inter-client data
heterogeneity, we note that intra-client heterogeneity can also be observed on
complex real-world data and seriously deteriorate FL performance. In this
paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client
data heterogeneity by enabling instance-adaptive inference in the FL framework.
Instead of huge instance-adaptive models, we resort to a parameter-efficient
fine-tuning method, i.e., scale and shift deep features (SSF), upon a
pre-trained model. Specifically, we first train an SSF pool for each client,
and aggregate these SSF pools on the server side, thus still maintaining a low
communication cost. To enable instance-adaptive inference, for a given
instance, we dynamically find the best-matched SSF subsets from the pool and
aggregate them to generate an adaptive SSF specified for the instance, thereby
reducing the intra-client as well as the inter-client heterogeneity. Extensive
experiments show that our FedIns outperforms state-of-the-art FL algorithms,
e.g., a 6.64\% improvement against the top-performing method with less than
15\% communication cost on Tiny-ImageNet. Our code and models will be publicly
released.

该论文提出了一种新颖的 FL 算法（FedIns）来处理 FL 框架中的内部客户数据异质性，并通过在 FL 框架中实现实例自适应推理来减少内部客户和跨客户之间的异质性，实验证明 FedIns 在 Tiny-ImageNet 上比现有 FL 算法表现更好（多于 6.64% 的性能提升，且通信成本不到 15%），代码和模型将在公开发布。

面向实例自适应的联邦学习推理

Towards Instance-adaptive Inference for Federated Learning

Graph Neural Network (GNN) research is rapidly growing thanks to the capacity
of GNNs in learning distributed representations from graph-structured data.
However, centralizing a massive amount of real-world graph data for GNN
training is prohibitive due to privacy concerns, regulation restrictions, and
commercial competitions. Federated learning (FL), a trending distributed
learning paradigm, provides possibilities to solve this challenge while
preserving data privacy. Despite recent advances in vision and language
domains, there is no suitable platform for the FL of GNNs. To this end, we
introduce FedGraphNN, an open FL benchmark system that can facilitate research
on federated GNNs. FedGraphNN is built on a unified formulation of graph FL and
contains a wide range of datasets from different domains, popular GNN models,
and FL algorithms, with secure and efficient system support. Particularly for
the datasets, we collect, preprocess, and partition 36 datasets from 7 domains,
including both publicly available ones and specifically obtained ones such as
hERG and Tencent. Our empirical analysis showcases the utility of our benchmark
system, while exposing significant challenges in graph FL: federated GNNs
perform worse in most datasets with a non-IID split than centralized GNNs; the
GNN model that attains the best result in the centralized setting may not
maintain its advantage in the FL setting. These results imply that more
research efforts are needed to unravel the mystery behind federated GNNs.
Moreover, our system performance analysis demonstrates that the FedGraphNN
system is computationally efficient and secure to large-scale graphs datasets.
We maintain the source code at this https URL

本研究介绍 FedGraphNN，一个用于解决中心化实际世界图形数据训练 GNN 所面临的隐私问题的开放式分布式学习平台，涵盖各种数据集，流行的 GNN 模型以及 FL 算法，维护源代码的统一公式和安全高效的系统支持。