In real-world applications, Federated Learning (FL) meets two challenges: (1)
scalability, especially when applied to massive IoT networks; and (2) how to be
robust against an environment with heterogeneous data. Realizing the first
problem, we aim to design a novel FL framework named Full-stack FL (F2L). More
specifically, F2L utilizes a hierarchical network architecture, making
extending the FL network accessible without reconstructing the whole network
system. Moreover, leveraging the advantages of hierarchical network design, we
propose a new label-driven knowledge distillation (LKD) technique at the global
server to address the second problem. As opposed to current knowledge
distillation techniques, LKD is capable of training a student model, which
consists of good knowledge from all teachers' models. Therefore, our proposed
algorithm can effectively extract the knowledge of the regions' data
distribution (i.e., the regional aggregated models) to reduce the divergence
between clients' models when operating under the FL system with non-independent
identically distributed data. Extensive experiment results reveal that: (i) our
F2L method can significantly improve the overall FL efficiency in all global
distillations, and (ii) F2L rapidly achieves convergence as global distillation
stages occur instead of increasing on each communication cycle.

提出一种名为 Full-stack FL 的新型联邦学习框架，通过使用分层网络架构和标签驱动的知识蒸馏技术，解决大规模 IoT 网络下的可扩展性问题和异构数据环境下的鲁棒性问题，从而在所有全局蒸馏中显著提高了联邦学习的效率。