The traditional approach in FL tries to learn a single global model
collaboratively with the help of many clients under the orchestration of a
central server. However, learning a single global model might not work well for
all clients participating in the FL under data heterogeneity. Therefore, the
personalization of the global model becomes crucial in handling the challenges
that arise with statistical heterogeneity and the non-IID distribution of data.
Unlike prior works, in this work we propose a new approach for obtaining a
personalized model from a client-level objective. This further motivates all
clients to participate in federation even under statistical heterogeneity in
order to improve their performance, instead of merely being a source of data
and model training for the central server. To realize this personalization, we
leverage finding a small subnetwork for each client by applying hybrid pruning
(combination of structured and unstructured pruning), and unstructured pruning.
Through a range of experiments on different benchmarks, we observed that the
clients with similar data (labels) share similar personal parameters. By
finding a subnetwork for each client ...

本文提出了一种针对统计异质性和非独立同分布数据的个性化全局模型联邦学习方法，使用混合剪枝和非结构化剪枝找到每个客户端的小型子网络来实现模型的个性化，实验结果表明，具有相似标签数据的客户端共享类似的个性化参数。

基于结构化和非结构化剪枝的数据异构下的个性化联邦学习

Personalized Federated Learning by Structured and Unstructured Pruning  under Data Heterogeneity

We introduce hybrid pruning which combines both coarse-grained channel and
fine-grained weight pruning to reduce model size, computation and power demands
with no to little loss in accuracy for enabling modern networks deployment on
resource-constrained devices, such as always-on security cameras and drones.
Additionally, to effectively perform channel pruning, we propose a fast
sensitivity test that helps us quickly identify the sensitivity of within and
across layers of a network to the output accuracy for target multiplier
accumulators (MACs) or accuracy tolerance. Our experiment shows significantly
better results on ResNet50 on ImageNet compared to existing work, even with an
additional constraint of channels be hardware-friendly number.

本文介绍了一种混合剪枝方法，将粗粒度通道和细粒度的权重剪枝相结合，以在资源受限设备上部署现代网络时减少模型大小、计算和功率需求，而无损或几乎无损精度的同时，提出了快速灵敏度测试的通道剪枝方法，可以快速确定网络层内部和跨层级的灵敏度，实验表明，即使在通道数硬件友好的约束下，相对于现有工作，ResNet50 在 ImageNet 上的结果显著更好。