Federated learning (FL) is a distributed learning paradigm that enables
multiple clients to learn a powerful global model by aggregating local
training. However, the performance of the global model is often hampered by
non-i.i.d. distribution among the clients, requiring extensive efforts to
mitigate inter-client data heterogeneity. Going beyond inter-client data
heterogeneity, we note that intra-client heterogeneity can also be observed on
complex real-world data and seriously deteriorate FL performance. In this
paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client
data heterogeneity by enabling instance-adaptive inference in the FL framework.
Instead of huge instance-adaptive models, we resort to a parameter-efficient
fine-tuning method, i.e., scale and shift deep features (SSF), upon a
pre-trained model. Specifically, we first train an SSF pool for each client,
and aggregate these SSF pools on the server side, thus still maintaining a low
communication cost. To enable instance-adaptive inference, for a given
instance, we dynamically find the best-matched SSF subsets from the pool and
aggregate them to generate an adaptive SSF specified for the instance, thereby
reducing the intra-client as well as the inter-client heterogeneity. Extensive
experiments show that our FedIns outperforms state-of-the-art FL algorithms,
e.g., a 6.64\% improvement against the top-performing method with less than
15\% communication cost on Tiny-ImageNet. Our code and models will be publicly
released.

该论文提出了一种新颖的 FL 算法（FedIns）来处理 FL 框架中的内部客户数据异质性，并通过在 FL 框架中实现实例自适应推理来减少内部客户和跨客户之间的异质性，实验证明 FedIns 在 Tiny-ImageNet 上比现有 FL 算法表现更好（多于 6.64% 的性能提升，且通信成本不到 15%），代码和模型将在公开发布。