Transformer-based large language models (LLMs) have demonstrated impressive
capabilities in a variety of natural language processing (NLP) tasks.
Nonetheless, it is challenging to deploy and fine-tune LLMs on mobile edge
devices with limited computing, memory, and energy budgets. In this paper, we
propose Confidant, a multi-backend collaborative training framework for
customizing state-of-the-art LLMs on commodity mobile devices like smartphones.
Confidant partitions an LLM into several sub-models so that each fits into a
mobile device's memory. A pipeline parallel training mechanism is further
developed to ensure fast and efficient distributed training. In addition, we
propose a novel backend scheduler to allocate different attention heads to
heterogeneous compute hardware, including mobile CPU and GPUs, to maximize the
compute resource utilization on each edge device. Our preliminary experimental
results show that Confidant achieves at most 45.3% memory reduction and 8.03x
inference speedup in practical settings.

为了在手机等无线设备上进行自定义最先进的语言模型（LLM）的训练，我们提出了 Confidant，这是一个多后端协同训练框架。Confidant 将 LLM 划分为几个子模型，以适应手机设备的内存，同时开发了一个管道并行训练机制以确保快速和高效的分布式训练。此外，我们提出了一种新颖的后端调度器，将不同的注意力头分配给包括手机 CPU 和 GPU 在内的异构计算硬件，以最大程度地提高每个边缘设备上的计算资源利用。我们的初步实验结果表明，Confidant 在实际环境中实现了最多 45.3% 的内存减少和 8.03 倍的推理加速。