In this work, we explored federated learning in temporal heterogeneity across
clients. We observed that global model obtained by \texttt{FedAvg} trained with
fixed-length sequences shows faster convergence than varying-length sequences.
We proposed methods to mitigate temporal heterogeneity for efficient federated
learning based on the empirical observation.

本研究探讨了在客户端之间存在的时间异质性的联邦学习，并观察到使用固定长度序列进行训练的 FedAvg 获得的全局模型比使用不同长度序列的模型收敛速度更快。我们提出了一种基于经验观察的方法来减轻时间异质性，以实现高效的联邦学习。

时序异质性中的联邦学习

Federated Learning in Temporal Heterogeneity

Personalized federated learning (PFL) aims to harness the collective wisdom
of clients' data to build customized models tailored to individual clients'
data distributions. Existing works offer personalization primarily to clients
who participate in the FL process, making it hard to encompass new clients who
were absent or newly show up. In this paper, we propose FedBasis, a novel PFL
framework to tackle such a deficiency. FedBasis learns a set of few, shareable
``basis'' models, which can be linearly combined to form personalized models
for clients. Specifically for a new client, only a small set of combination
coefficients, not the models, needs to be learned. This notion makes FedBasis
more parameter-efficient, robust, and accurate compared to other competitive
PFL baselines, especially in the low data regime, without increasing the
inference cost. To demonstrate its applicability, we also present a more
practical PFL testbed for image classification, featuring larger data
discrepancies across clients in both the image and label spaces as well as more
faithful training and test splits.

本文提出了一种新的个性化联邦学习框架 ——FedBasis，它学习一组共享的 “基础” 模型，可以线性组合形成客户定制的模型，从而实现在低数据情况下减少推断成本，同时提高参数效率和鲁棒性。

基于联邦学习的可共享特征训练方法，用于友好个性化图像分类

Federated Learning of Shareable Bases for Personalization-Friendly Image  Classification

Federated Learning aims at training a global model from multiple
decentralized devices (i.e. clients) without exchanging their private local
data. A key challenge is the handling of non-i.i.d. (independent identically
distributed) data across multiple clients that may induce disparities of their
local features. We introduce the Hyperspherical Federated Learning (SphereFed)
framework to address the non-i.i.d. issue by constraining learned
representations of data points to be on a unit hypersphere shared by clients.
Specifically, all clients learn their local representations by minimizing the
loss with respect to a fixed classifier whose weights span the unit
hypersphere. After federated training in improving the global model, this
classifier is further calibrated with a closed-form solution by minimizing a
mean squared loss. We show that the calibration solution can be computed
efficiently and distributedly without direct access of local data. Extensive
experiments indicate that our SphereFed approach is able to improve the
accuracy of multiple existing federated learning algorithms by a considerable
margin (up to 6% on challenging datasets) with enhanced computation and
communication efficiency across datasets and model architectures.

使用球面限制措施的超球面联邦学习 (SphereFed) 框架，通过在联邦学习之前将数据点的表示限制到由客户端共享的单位超球面上，解决了非独立分布数据的处理问题，通过构建固定分类器来处理不类别分布的数据在不同客户端间的差异性并在最小化损失函数的过程中进行联邦学习，最终实现进一步的分类器校准。我们的实验证明 SphereFed 方法能够有效提高现有算法的准确性（在具有挑战性的数据集上达到 6% 以上），并增强计算和通信效率。