Many recent studies integrate federated learning (FL) with self-supervised
learning (SSL) to take advantage of raw training data distributed across edge
devices. However, edge devices often struggle with high computation and
communication costs imposed by SSL and FL algorithms. To tackle this hindrance,
we propose LW-FedSSL, a layer-wise federated self-supervised learning approach
that allows edge devices to incrementally train one layer of the model at a
time. LW-FedSSL comprises server-side calibration and representation alignment
mechanisms to maintain comparable performance with end-to-end FedSSL while
significantly lowering clients' resource requirements. The server-side
calibration mechanism takes advantage of the resource-rich server in an FL
environment to assist in global model training. Meanwhile, the representation
alignment mechanism encourages closeness between representations of FL local
models and those of the global model. Our experiments show that LW-FedSSL has a
$3.3 \times$ lower memory requirement and a $3.2 \times$ cheaper communication
cost than its end-to-end counterpart. We also explore a progressive training
strategy called Prog-FedSSL that outperforms end-to-end training with a similar
memory requirement and a $1.8 \times$ cheaper communication cost.

提出一种层次化的联邦自监督学习方法 (LW-FedSSL)，通过在 FL 环境中利用资源丰富的服务器进行全局模型训练，以及在 FL 本地模型与全局模型之间建立紧密联系的表示对齐机制，实现了与端到端 FedSSL 相当的性能，同时显著降低客户端的资源需求。实验证明 LW-FedSSL 的内存需求较其端到端对应物少 3.3 倍，并且通信成本便宜 3.2 倍。同时，还探索了一种名为 Prog-FedSSL 的渐进训练策略，其在类似内存需求的情况下，通信成本便宜 1.8 倍，优于端到端训练。