Federated Learning (FL) is becoming a popular paradigm for leveraging
distributed data and preserving data privacy. However, due to the distributed
characteristic, FL systems are vulnerable to Byzantine attacks that compromised
clients attack the global model by uploading malicious model updates. Most
existing Byzantine-robust FL systems statistically analyze the weights of whole
individual model updates uploaded by clients to defend against Byzantine
attacks. With the development of layer-level and parameter-level fine-grained
attacks, the attacks' stealthiness and effectiveness have been significantly
improved. Due to unawareness or overreaction, the existing model-level defense
methods degrade the training efficiency and model performance. To address this
problem, we propose SkyMask, a new attack-agnostic robust FL system that
leverages fine-grained learnable masks to identify malicious model updates at
the parameter-level. Specifically, the FL server applies parameter-level masks
to model updates uploaded by clients and trains the masks over a small clean
dataset (i.e., root dataset) to learn the subtle difference between benign and
malicious model updates in a high-dimension space. Our extensive experiments
involve different models on three public datasets under state-of-the-art (SOTA)
attacks, where the results show that SkyMask achieves up to 10% higher testing
accuracy compared with SOTA defense strategies and successfully defends against
attacks with malicious clients of a high fraction up to 80%. In the meantime,
the experimental results demonstrate the scalability of our approach and the
weak dependence on the data distribution of the root dataset.

提出了一种新的攻击不可知的鲁棒联邦学习系统 SkyMask，它利用细粒度的可学习掩码在参数级别上识别恶意模型更新，以解决现有防御方法的低效率和模型性能下降的问题。