Due to user privacy and regulatory restrictions, federate learning (FL) is
proposed as a distributed learning framework for training deep neural networks
(DNN) on decentralized data clients. Recent advancements in FL have applied
Neural Architecture Search (NAS) to replace the predefined one-size-fit-all DNN
model, which is not optimal for all tasks of various data distributions, with
searchable DNN architectures. However, previous methods suffer from expensive
communication cost rasied by frequent large model parameters transmission
between the server and clients. Such difficulty is further amplified when
combining NAS algorithms, which commonly require prohibitive computation and
enormous model storage. Towards this end, we propose FAQS, an efficient
personalized FL-NAS-Quantization framework to reduce the communication cost
with three features: weight-sharing super kernels, bit-sharing quantization and
masked transmission. FAQS has an affordable search time and demands very
limited size of transmitted messages at each round. By setting different
personlized pareto function loss on local clients, FAQS can yield heterogeneous
hardware-aware models for various user preferences. Experimental results show
that FAQS achieves average reduction of 1.58x in communication bandwith per
round compared with normal FL framework and 4.51x compared with FL+NAS
framwork.

提出了 FAQS，这是一个高效的个性化 FL-NAS 量化框架，具有权重共享超级内核，比特共享量化和掩码传输等特征，可帮助减少通信成本并生成适合于各种用户偏好的异构硬件感知模型。实验结果表明，FAQS 比常规 FL 框架每轮通信带宽平均降低 1.58 倍，并比 FL+NAS 框架高效 4.51 倍。