Federated Learning (FL) is a prominent distributed learning paradigm
facilitating collaboration among nodes within an edge network to co-train a
global model without centralizing data. By shifting computation to the network
edge, FL offers robust and responsive edge-AI solutions and enhance
privacy-preservation. However, deploying deep FL models within edge
environments is often hindered by communication bottlenecks, data
heterogeneity, and memory limitations. To address these challenges jointly, we
introduce FeDEQ, a pioneering FL framework that effectively employs deep
equilibrium learning and consensus optimization to exploit a compact shared
data representation across edge nodes, allowing the derivation of personalized
models specific to each node. We delve into a unique model structure composed
of an equilibrium layer followed by traditional neural network layers. Here,
the equilibrium layer functions as a global feature representation that edge
nodes can adapt to personalize their local layers. Capitalizing on FeDEQ's
compactness and representation power, we present a novel distributed algorithm
rooted in the alternating direction method of multipliers (ADMM) consensus
optimization and theoretically establish its convergence for smooth objectives.
Experiments across various benchmarks demonstrate that FeDEQ achieves
performance comparable to state-of-the-art personalized methods while employing
models of up to 4 times smaller in communication size and 1.5 times lower
memory footprint during training.

FeDEQ 是一种先驱性的联邦学习框架，利用深度均衡学习和一致性优化有效地在边缘节点之间利用紧凑的共享数据表示，允许派生个性化模型，并通过采用交替方向法解（ADMM）一致性优化的新型分布式算法，理论上证明了其收敛性。实验结果表明，与最先进的个性化方法相比，FeDEQ 在通信大小上使用的模型可小 4 倍，并且在训练时内存占用更少 1.5 倍，性能可媲美。