Large language models are rapidly gaining popularity and have been widely adopted in real-world applications. While the quality of training data is essential, privacy concerns arise during data collection. Federated learning offers a solution by allowing multiple clients to collaboratively train LLMs without sharing local data. However, FL introduces new challenges, such as model convergence issues due to heterogeneous data and high communication costs. A comprehensive study is required to address these challenges and guide future research. This paper surveys Federated learning for LLMs (FedLLM), highlighting recent advances and future directions. We focus on two key aspects: fine-tuning and prompt learning in a federated setting, discussing existing work and associated research challenges. We finally propose potential research directions for federated LLMs, including pre-training and how LLMs can further enhance federated learning.

该研究解决了在数据收集过程中由于隐私问题导致的训练数据质量担忧。论文调研了联邦学习在大型语言模型中的应用，重点探讨了在联邦设置下细化和提示学习的研究挑战和现有工作，同时提出了未来研究方向，旨在提高模型的收敛性及降低通信成本。

联邦大型语言模型：当前进展与未来方向