The large communication and computation overhead of federated learning (FL)
is one of the main challenges facing its practical deployment over
resource-constrained clients and systems. In this work, SpaFL: a
communication-efficient FL framework is proposed to optimize sparse model
structures with low computational overhead. In SpaFL, a trainable threshold is
defined for each filter/neuron to prune its all connected parameters, thereby
leading to structured sparsity. To optimize the pruning process itself, only
thresholds are communicated between a server and clients instead of parameters,
thereby learning how to prune. Further, global thresholds are used to update
model parameters by extracting aggregated parameter importance. The
generalization bound of SpaFL is also derived, thereby proving key insights on
the relation between sparsity and performance. Experimental results show that
SpaFL improves accuracy while requiring much less communication and computing
resources compared to sparse baselines.

SpaFL 提出了一个通信高效的联邦学习框架，优化了稀疏模型结构来避免大规模通信和计算资源消耗，通过定义可训练的阈值来剪枝连接参数以实现结构化稀疏性，只通过服务器和客户端之间的阈值通信来学习如何剪枝，利用全局阈值提取聚合参数重要性来更新模型参数，并通过推导 SpaFL 的泛化界限，证明了稀疏性与性能之间的关系，实验结果表明 SpaFL 在准确性方面改进的同时相比稀疏基准需要更少的通信和计算资源。