Federated Learning (FL) is a privacy preserving machine learning scheme,
where training happens with data federated across devices and not leaving them
to sustain user privacy. This is ensured by making the untrained or partially
trained models to reach directly the individual devices and getting locally
trained "on-device" using the device owned data, and the server aggregating all
the partially trained model learnings to update a global model. Although almost
all the model learning schemes in the federated learning setup use gradient
descent, there are certain characteristic differences brought about by the
non-IID nature of the data availability, that affects the training in
comparison to the centralized schemes. In this paper, we discuss the various
factors that affect the federated learning training, because of the non-IID
distributed nature of the data, as well as the inherent differences in the
federating learning approach as against the typical centralized gradient
descent techniques. We empirically demonstrate the effect of number of samples
per device and the distribution of output labels on federated learning. In
addition to the privacy advantage we seek through federated learning, we also
study if there is a cost advantage while using federated learning frameworks.
We show that federated learning does have an advantage in cost when the model
sizes to be trained are not reasonably large. All in all, we present the need
for careful design of model for both performance and cost.

本文讨论分布式数据的非 IID 性质及其对联邦学习的训练造成的影响，同时对联邦学习与典型集中式梯度下降技术的不同之处进行比较，并在实践中演示样品数量和输出标签分布对联邦学习的影响。此外，本文证明了在模型大小不太大的情况下使用联邦学习框架具有成本优势。