The fast-growing large scale language models are delivering unprecedented
performance on almost all natural language processing tasks. However, the
effectiveness of large language models are reliant on an exponentially
increasing number of parameters. The overwhelming computation complexity incurs
a high inference latency that negatively affects user experience. Existing
methods to improve inference efficiency, such as tensor parallelism and
quantization, target to reduce per-layer computing latency, yet overlook the
cumulative latency due to the number of layers. Recent works on reducing the
cumulative latency through layer removing, however, lead to significant
performance drop. Motivated by the similarity of inputs among adjacent layers,
we propose to identify quasi-independent layers, which can be concurrently
computed to significantly decrease inference latency. We also introduce a
bypassing technique to mitigate the effect of information loss. Empirical
experiments of the proposed approach on the LLaMA models confirm that
Concurrent Computation of Quasi-Independent Layers (CQIL) can reduce latency by
up to 48.3% on the LLaMA-33B model, while maintaining a close level of
performance.

大规模语言模型中的并行计算减少推理延迟，提高性能。

CQIL：并发计算准独立层优化推理延迟

CQIL: Inference Latency Optimization with Concurrent Computation of  Quasi-Independent Layers

Large scale language models (LLM) have received significant attention and
found diverse applications across various domains, but their development
encounters challenges in real-world scenarios. These challenges arise due to
the scarcity of public domain data availability and the need to maintain
privacy with respect to private domain data. To address these issues, federated
learning (FL) has emerged as a promising technology that enables collaborative
training of shared models while preserving decentralized data. We propose the
concept of federated LLM, which comprises three key components, i.e., federated
LLM pre-training, federated LLM fine-tuning, and federated LLM prompt
engineering. For each component, we discuss its advantage over traditional LLM
training methods and propose specific engineering strategies for
implementation. Furthermore, we explore the novel challenges introduced by the
integration of FL and LLM. We analyze existing solutions and identify potential
obstacles faced by these solutions within the context of federated LLM.

提出了联邦式大规模语言模型（LLM）的概念，并讨论了其三个关键组成部分：联邦式 LLM 预训练、联邦式 LLM 微调和联邦式 LLM 提示工程。还探讨了联邦学习和大规模语言模型整合所带来的新挑战，并分析了现有解决方案和潜在障碍。