Data heterogeneity presents significant challenges for federated learning
(FL). Recently, dataset distillation techniques have been introduced, and
performed at the client level, to attempt to mitigate some of these challenges.
In this paper, we propose a highly efficient FL dataset distillation framework
on the server side, significantly reducing both the computational and
communication demands on local devices while enhancing the clients' privacy.
Unlike previous strategies that perform dataset distillation on local devices
and upload synthetic data to the server, our technique enables the server to
leverage prior knowledge from pre-trained deep generative models to synthesize
essential data representations from a heterogeneous model architecture. This
process allows local devices to train smaller surrogate models while enabling
the training of a larger global model on the server, effectively minimizing
resource utilization. We substantiate our claim with a theoretical analysis,
demonstrating the asymptotic resemblance of the process to the hypothetical
ideal of completely centralized training on a heterogeneous dataset. Empirical
evidence from our comprehensive experiments indicates our method's superiority,
delivering an accuracy enhancement of up to 40% over non-dataset-distillation
techniques in highly heterogeneous FL contexts, and surpassing existing
dataset-distillation methods by 18%. In addition to the high accuracy, our
framework converges faster than the baselines because rather than the server
trains on several sets of heterogeneous data distributions, it trains on a
multi-modal distribution. Our code is available at
this https URL

基于深度生成模型的高效联邦学习数据集精简框架，在服务器端显著减少本地设备的计算和通信需求，提高客户端隐私，同时有效降低资源利用，通过理论分析和实证证据验证了其在高度异构的联邦学习环境中相较于非数据集精简技术的优越性，准确性提升高达 40%，超过已有的数据集精简方法 18%。