The industrial Internet of Things (IIoT) under Industry 4.0 heralds an era of
interconnected smart devices where data-driven insights and machine learning
(ML) fuse to revolutionize manufacturing. A noteworthy development in IIoT is
the integration of federated learning (FL), which addresses data privacy and
security among devices. FL enables edge sensors, also known as peripheral
intelligence units (PIUs) to learn and adapt using their data locally, without
explicit sharing of confidential data, to facilitate a collaborative yet
confidential learning process. However, the lower memory footprint and
computational power of PIUs inherently require deep neural network (DNN) models
that have a very compact size. Model compression techniques such as pruning can
be used to reduce the size of DNN models by removing unnecessary connections
that have little impact on the model's performance, thus making the models more
suitable for the limited resources of PIUs. Targeting the notion of compact yet
robust DNN models, we propose the integration of iterative magnitude pruning
(IMP) of the DNN model being trained in an over-the-air FL (OTA-FL) environment
for IIoT. We provide a tutorial overview and also present a case study of the
effectiveness of IMP in OTA-FL for an IIoT environment. Finally, we present
future directions for enhancing and optimizing these deep compression
techniques further, aiming to push the boundaries of IIoT capabilities in
acquiring compact yet robust and high-performing DNN models.

工业物联网（IIoT）在第四次工业革命下开启了一个互连的智能设备时代，其中数据驱动的洞察力和机器学习（ML）融合革新了制造业。在 IIoT 中，一个值得注意的发展是联邦学习（FL）的整合，解决了设备间的数据隐私和安全性。FL 使得边缘传感器（也称为周边智能单元）能够在本地使用数据进行学习和适应，而无需明确共享机密数据，从而促进了一种协作但保密的学习过程。然而，边缘智能单元内存占用和计算能力的限制要求使用非常紧凑的深度神经网络（DNN）模型。我们提出将经过迭代幅度剪枝（IMP）的 DNN 模型整合到用于 IIoT 的 OTA-FL 环境中，以实现紧凑而强大的模型。我们提供了教程概述，并针对 IIoT 环境中 OTA-FL 的有效性进行了案例研究。最后，我们提出了进一步增强和优化这些深度压缩技术的未来方向，旨在推动 IIoT 能力的边界，实现紧凑且强大的高性能 DNN 模型的获取。

采用迭代度量剪枝推进工业物联网的 OTA 联邦学习

Advancing IIoT with Over-the-Air Federated Learning: The Role of  Iterative Magnitude Pruning

Addressing the communication bottleneck inherent in federated learning (FL),
over-the-air FL (AirFL) has emerged as a promising solution, which is, however,
hampered by deep fading conditions. In this paper, we propose AirFL-Mem, a
novel scheme designed to mitigate the impact of deep fading by implementing a
\emph{long-term} memory mechanism. Convergence bounds are provided that account
for long-term memory, as well as for existing AirFL variants with short-term
memory, for general non-convex objectives. The theory demonstrates that
AirFL-Mem exhibits the same convergence rate of federated averaging (FedAvg)
with ideal communication, while the performance of existing schemes is
generally limited by error floors. The theoretical results are also leveraged
to propose a novel convex optimization strategy for the truncation threshold
used for power control in the presence of Rayleigh fading channels.
Experimental results validate the analysis, confirming the advantages of a
long-term memory mechanism for the mitigation of deep fading.

提出了一种名为 AirFL-Mem 的新方案来缓解深衰落对过空中远程联邦学习的影响，并证明了它与理想通信条件下的联邦平均算法 (FedAvg) 具有相同的收敛速度，而现有方案的性能通常受到误差限制。实验结果验证了长期记忆机制在缓解深衰落方面的优势。

AirFL-Mem: 通过长期记忆改进通信 - 学习的权衡

AirFL-Mem: Improving Communication-Learning Trade-Off by Long-Term  Memory

Federated Learning (FL) is a widely embraced paradigm for distilling
artificial intelligence from distributed mobile data. However, the deployment
of FL in mobile networks can be compromised by exposure to interference from
neighboring cells or jammers. Existing interference mitigation techniques
require multi-cell cooperation or at least interference channel state
information, which is expensive in practice. On the other hand, power control
that treats interference as noise may not be effective due to limited power
budgets, and also that this mechanism can trigger countermeasures by
interference sources. As a practical approach for protecting FL against
interference, we propose Spectrum Breathing, which cascades stochastic-gradient
pruning and spread spectrum to suppress interference without bandwidth
expansion. The cost is higher learning latency by exploiting the graceful
degradation of learning speed due to pruning. We synchronize the two operations
such that their levels are controlled by the same parameter, Breathing Depth.
To optimally control the parameter, we develop a martingale-based approach to
convergence analysis of Over-the-Air FL with spectrum breathing, termed
AirBreathing FL. We show a performance tradeoff between gradient-pruning and
interference-induced error as regulated by the breathing depth. Given receive
SIR and model size, the optimization of the tradeoff yields two schemes for
controlling the breathing depth that can be either fixed or adaptive to
channels and the learning process. As shown by experiments, in scenarios where
traditional Over-the-Air FL fails to converge in the presence of strong
interference, AirBreahing FL with either fixed or adaptive breathing depth can
ensure convergence where the adaptive scheme achieves close-to-ideal
performance.

该研究提出了一种名为 Spectrum Breathing 的方法用于保护在移动网络中分布式学习模型的有效性并降低干扰，该方法整合了随机梯度剪枝和扩频技术来达到最佳控制参数并实现收敛分析，在实验中表现出良好的性能。