Large language models (LLMs), which have shown remarkable capabilities, are
revolutionizing AI development and potentially shaping our future. However,
given their multimodality, the status quo cloud-based deployment faces some
critical challenges: 1) long response time; 2) high bandwidth costs; and 3) the
violation of data privacy. 6G mobile edge computing (MEC) systems may resolve
these pressing issues. In this article, we explore the potential of deploying
LLMs at the 6G edge. We start by introducing killer applications powered by
multimodal LLMs, including robotics and healthcare, to highlight the need for
deploying LLMs in the vicinity of end users. Then, we identify the critical
challenges for LLM deployment at the edge and envision the 6G MEC architecture
for LLMs. Furthermore, we delve into two design aspects, i.e., edge training
and edge inference for LLMs. In both aspects, considering the inherent resource
limitations at the edge, we discuss various cutting-edge techniques, including
split learning/inference, parameter-efficient fine-tuning, quantization, and
parameter-sharing inference, to facilitate the efficient deployment of LLMs.
This article serves as a position paper for thoroughly identifying the
motivation, challenges, and pathway for empowering LLMs at the 6G edge.

大型语言模型的部署面临一些挑战，云部署方式会有长时间响应、高带宽成本和数据隐私问题。文章探讨了在 6G 边缘部署语言模型的潜力，介绍了多模态语言模型的关键应用，并提出了 6G 移动边缘计算的架构。此外，讨论了边缘训练和边缘推理的设计方面，提出了一些有效的技术以促进语言模型的高效部署。

推动大型语言模型走向 6G 边缘：愿景、挑战和机遇

Pushing Large Language Models to the 6G Edge: Vision, Challenges, and  Opportunities

Edge training of Deep Neural Networks (DNNs) is a desirable goal for
continuous learning; however, it is hindered by the enormous computational
power required by training. Hardware approximate multipliers have shown their
effectiveness for gaining resource-efficiency in DNN inference accelerators;
however, training with approximate multipliers is largely unexplored. To build
resource efficient accelerators with approximate multipliers supporting DNN
training, a thorough evaluation of training convergence and accuracy for
different DNN architectures and different approximate multipliers is needed.
This paper presents ApproxTrain, an open-source framework that allows fast
evaluation of DNN training and inference using simulated approximate
multipliers. ApproxTrain is as user-friendly as TensorFlow (TF) and requires
only a high-level description of a DNN architecture along with C/C++ functional
models of the approximate multiplier. We improve the speed of the simulation at
the multiplier level by using a novel LUT-based approximate floating-point (FP)
multiplier simulator on GPU (AMSim). ApproxTrain leverages CUDA and efficiently
integrates AMSim into the TensorFlow library, in order to overcome the absence
of native hardware approximate multiplier in commercial GPUs. We use
ApproxTrain to evaluate the convergence and accuracy of DNN training with
approximate multipliers for small and large datasets (including ImageNet) using
LeNets and ResNets architectures. The evaluations demonstrate similar
convergence behavior and negligible change in test accuracy compared to FP32
and bfloat16 multipliers. Compared to CPU-based approximate multiplier
simulations in training and inference, the GPU-accelerated ApproxTrain is more
than 2500x faster. Based on highly optimized closed-source cuDNN/cuBLAS
libraries with native hardware multipliers, the original TensorFlow is only 8x
faster than ApproxTrain.

本文通过开源框架 ApproxTrain，使用模拟的近似乘法器评估了使用近似乘法器的深度神经网络（DNN）培训的收敛性和准确性，论证了这种方法的资源效率和 GPU 加速优势。