BriefGPT.xyz
Jun, 2022
从预训练和初始化对联邦学习的影响开始
Where to Begin? Exploring the Impact of Pre-Training and Initialization in Federated Learning
HTML
PDF
John Nguyen, Kshitiz Malik, Maziar Sanjabi, Michael Rabbat
TL;DR
研究表明:使用预训练模型可以减少联邦学习中由数据和系统异构性带来的影响,缩短训练时间并提高模型准确性。同时,作者建议未来研究需要考虑从随机和预训练初始化开始时使用联邦优化方法的性能。
Abstract
An oft-cited challenge of
federated learning
is the presence of data
heterogeneity
-- the data at different clients may follow very different distributions. Several federated
→