In a connection of many IoT devices that each collect data, normally training
a machine learning model would involve transmitting the data to a central
server which requires strict privacy rules. However, some owners are reluctant
of availing their data out of the company due to data security concerns.
Federated learning(FL) as a distributed machine learning approach performs
training of a machine learning model on the device that gathered the data
itself. In this scenario, data is not share over the network for training
purpose. Fedavg as one of FL algorithms permits a model to be copied to
participating devices during a training session. The devices could be chosen at
random, and a device can be aborted. The resulting models are sent to the
coordinating server and then average models from the devices that finished
training. The process is repeated until a desired model accuracy is achieved.
By doing this, FL approach solves the privacy problem for IoT/ IIoT devices
that held sensitive data for the owners. In this paper, we leverage the
benefits of FL and implemented Fedavg algorithm on a recent dataset that
represent the modern IoT/ IIoT device networks. The results were almost the
same as the centralized machine learning approach. We also evaluated some
shortcomings of Fedavg such as unfairness that happens during the training when
struggling devices do not participate for every stage of training. This
inefficient training of local or global model could lead in a high number of
false alarms in intrusion detection systems for IoT/IIoT gadgets developed
using Fedavg. Hence, after evaluating the FedAv deep auto encoder with
centralized deep auto encoder ML, we further proposed and designed a Fair
Fedavg algorithm that will be evaluated in the future work.

通过使用联邦学习方法中的 Fedavg 算法解决了物联网设备中涉及的隐私问题，并提出了一个公平的 Fedavg 算法以解决在训练过程中出现的不公平问题。

利用联邦学习进行工业物联网边缘设备网络的无监督异常检测

Unsupervised anomalies detection in IIoT edge devices networks using  federated learning

The Federated Averaging (FedAvg) algorithm, which consists of alternating
between a few local stochastic gradient updates at client nodes, followed by a
model averaging update at the server, is perhaps the most commonly used method
in Federated Learning. Notwithstanding its simplicity, several empirical
studies have illustrated that the output model of FedAvg, after a few
fine-tuning steps, leads to a model that generalizes well to new unseen tasks.
This surprising performance of such a simple method, however, is not fully
understood from a theoretical point of view. In this paper, we formally
investigate this phenomenon in the multi-task linear representation setting. We
show that the reason behind generalizability of the FedAvg's output is its
power in learning the common data representation among the clients' tasks, by
leveraging the diversity among client data distributions via local updates. We
formally establish the iteration complexity required by the clients for proving
such result in the setting where the underlying shared representation is a
linear map. To the best of our knowledge, this is the first such result for any
setting. We also provide empirical evidence demonstrating FedAvg's
representation learning ability in federated image classification with
heterogeneous data.

通过对 FedAvg 在多任务线性表示设置下的形式化研究，我们发现 FedAvg 之所以能够产生良好的泛化效果，是因为能够通过本地更新利用客户数据分布之间的差异性来学习客户任务的公共数据表示，我们还提供了实证证据，证明了 FedAvg 在具有异构数据的联合图像分类中的表示学习能力。

本地更新促进表示学习的 FedAvg 与微调

FedAvg with Fine Tuning: Local Updates Lead to Representation Learning

Federated learning involves a mixture of centralized and decentralized
processing tasks, where a server regularly selects a sample of the agents and
these in turn sample their local data to compute stochastic gradients for their
learning updates. This process runs continually. The sampling of both agents
and data is generally uniform; however, in this work we consider non-uniform
sampling. We derive optimal importance sampling strategies for both agent and
data selection and show that non-uniform sampling without replacement improves
the performance of the original FedAvg algorithm. We run experiments on a
regression and classification problem to illustrate the theoretical results.

本论文研究在联邦学习中使用非均匀的重要性采样策略，证明此策略能够提高算法性能，并在回归和分类问题上进行实验来说明理论结果。

联邦学习的最优重要性采样

Optimal Importance Sampling for Federated Learning

Federated Learning enables visual models to be trained on-device, bringing
advantages for user privacy (data need never leave the device), but challenges
in terms of data diversity and quality. Whilst typical models in the datacenter
are trained using data that are independent and identically distributed (IID),
data at source are typically far from IID. Furthermore, differing quantities of
data are typically available at each device (imbalance). In this work, we
characterize the effect these real-world data distributions have on distributed
learning, using as a benchmark the standard Federated Averaging (FedAvg)
algorithm. To do so, we introduce two new large-scale datasets for species and
landmark classification, with realistic per-user data splits that simulate
real-world edge learning scenarios. We also develop two new algorithms (FedVC,
FedIR) that intelligently resample and reweight over the client pool, bringing
large improvements in accuracy and stability in training. The datasets are made
available online.

本研究探讨如何在真实世界的场景下训练视觉模型。我们使用了两个新的大规模数据集，测试了 Federated Learning 算法的表现，并提出了两种新的算法以优化模型的表现。