Batch normalization has proven to be a very beneficial mechanism to
accelerate the training and improve the accuracy of deep neural networks in
centralized environments. Yet, the scheme faces significant challenges in
federated learning, especially under high data heterogeneity. Essentially, the
main challenges arise from external covariate shifts and inconsistent
statistics across clients. We introduce in this paper Federated BatchNorm
(FBN), a novel scheme that restores the benefits of batch normalization in
federated learning. Essentially, FBN ensures that the batch normalization
during training is consistent with what would be achieved in a centralized
execution, hence preserving the distribution of the data, and providing running
statistics that accurately approximate the global statistics. FBN thereby
reduces the external covariate shift and matches the evaluation performance of
the centralized setting. We also show that, with a slight increase in
complexity, we can robustify FBN to mitigate erroneous statistics and
potentially adversarial attacks.

在联合学习中，批量标准化 (Federated BatchNorm) 是一种新的机制，它在训练中保持一致性，确保批量标准化与中心化执行中所获得的一致，并准确估计全局统计数据，从而减少外部协变量转移和与中心化设置的评估性能匹配。此外，可以通过稍微增加复杂性来加强 FBN 以减轻错误统计和可能的对抗攻击。