In the evolution towards 6G, integrating Artificial Intelligence (AI) with
advanced network infrastructure emerges as a pivotal strategy for enhancing
network intelligence and resource utilization. Existing distributed learning
frameworks like Federated Learning and Split Learning often struggle with
significant challenges in dynamic network environments including high
synchronization demands, costly communication overheads, severe computing
resource consumption, and data heterogeneity across network nodes. These
obstacles hinder the applications of ubiquitous computing capabilities of 6G
networks, especially in light of the trend of escalating model parameters and
training data volumes. To address these challenges effectively, this paper
introduces "Snake Learning", a cost-effective distributed learning framework.
Specifically, Snake Learning respects the heterogeneity of inter-node computing
capability and local data distribution in 6G networks, and sequentially trains
the designated part of model layers on individual nodes. This layer-by-layer
serpentine update mechanism contributes to significantly reducing the
requirements for storage, memory and communication during the model training
phase, and demonstrates superior adaptability and efficiency for both Computer
Vision (CV) training and Large Language Model (LLM) fine-tuning tasks across
homogeneous and heterogeneous data distributions.

本研究介绍了一种成本效益高的分布式学习框架 Snake Learning，它在 6G 网络中考虑了计算能力的异构性和本地数据分布，通过逐层更新机制显著降低了存储、内存和通信需求，在计算机视觉和大型语言模型任务中展现了出色的适应性和效率。