While large language models (LLMs) are empowered with broad knowledge, their
task-specific performance is often suboptimal. It necessitates fine-tuning LLMs
with task-specific data, but such data may be inaccessible due to privacy
concerns. In this paper, we propose a novel approach to enhance LLMs with
smaller language models (SLMs) that are trained on clients using their private
task-specific data. To enable mutual enhancement between LLMs and SLMs, we
propose CrossLM, where the SLMs promote the LLM to generate task-specific
high-quality data, and both the LLM and SLMs are enhanced with the generated
data. We evaluate CrossLM using publicly accessible language models across a
range of benchmark tasks. The results demonstrate that CrossLM significantly
enhances the task-specific performance of SLMs on clients and the LLM on the
cloud server simultaneously while preserving the LLM's generalization
capability.

提出了一种利用小规模语言模型与大规模语言模型相互提升的方法，通过小规模语言模型生成特定任务的高质量数据，从而显著提高客户端的小规模语言模型和云服务器的大规模语言模型的特定任务性能，同时保持大规模语言模型的泛化能力。