Federated learning (FL) is a powerful machine learning paradigm which
leverages the data as well as the computational resources of clients, while
protecting clients' data privacy. However, the substantial model size and
frequent aggregation between the server and clients result in significant
communication overhead, making it challenging to deploy FL in resource-limited
wireless networks. In this work, we aim to mitigate the communication overhead
by using quantization. Previous research on quantization has primarily focused
on the uplink communication, employing either fixed-bit quantization or
adaptive quantization methods. In this work, we introduce a holistic approach
by joint uplink and downlink adaptive quantization to reduce the communication
overhead. In particular, we optimize the learning convergence by determining
the optimal uplink and downlink quantization bit-length, with a communication
energy constraint. Theoretical analysis shows that the optimal quantization
levels depend on the range of model gradients or weights. Based on this
insight, we propose a decreasing-trend quantization for the uplink and an
increasing-trend quantization for the downlink, which aligns with the change of
the model parameters during the training process. Experimental results show
that, the proposed joint uplink and downlink adaptive quantization strategy can
save up to 66.7% energy compared with the existing schemes.

通过使用量化的综合方法，联合上下行适应性量化以减少通信开销，我们优化了学习收敛性，并通过确定最优的上行和下行量化位数进行了通信能量约束。实验结果表明，所提出的联合上行和下行适应性量化策略与现有方案相比，能够节省高达 66.7% 的能量。

FedAQ: 基于上下行自适应量化的通信高效联邦边缘学习

FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink  and Downlink Adaptive Quantization

Many DNN-enabled vision applications constantly operate under severe energy
constraints such as unmanned aerial vehicles, Augmented Reality headsets, and
smartphones. Designing DNNs that can meet a stringent energy budget is becoming
increasingly important. This paper proposes ECC, a framework that compresses
DNNs to meet a given energy constraint while minimizing accuracy loss. The key
idea of ECC is to model the DNN energy consumption via a novel bilinear
regression function. The energy estimate model allows us to formulate DNN
compression as a constrained optimization that minimizes the DNN loss function
over the energy constraint. The optimization problem, however, has nontrivial
constraints. Therefore, existing deep learning solvers do not apply directly.
We propose an optimization algorithm that combines the essence of the
Alternating Direction Method of Multipliers (ADMM) framework with
gradient-based learning algorithms. The algorithm decomposes the original
constrained optimization into several subproblems that are solved iteratively
and efficiently. ECC is also portable across different hardware platforms
without requiring hardware knowledge. Experiments show that ECC achieves higher
accuracy under the same or lower energy budget compared to state-of-the-art
resource-constrained DNN compression techniques.

本文提出了 ECC 框架，它使用双线性回归函数模拟 DNN 的能耗，将 DNN 压缩到给定的能量限制下，并通过结合 ADMM 框架和梯度下降算法的方法优化约束优化问题，从而实现在相同或更低的能量预算下实现更高的精度。