While training large language models (LLMs) from scratch can indeed lead to
models with distinct capabilities and strengths, this approach incurs
substantial costs and may lead to potential redundancy in competencies. An
alternative strategy is to combine existing LLMs into a more robust LLM,
thereby diminishing the necessity for expensive pre-training. However, due to
the diverse architectures of LLMs, direct parameter blending proves to be
unfeasible. Recently, \textsc{FuseLLM} introduced the concept of knowledge
fusion to transfer the collective knowledge of multiple structurally varied
LLMs into a target LLM through lightweight continual training. In this report,
we extend the scalability and flexibility of the \textsc{FuseLLM} framework to
realize the fusion of chat LLMs, resulting in \textsc{FuseChat}.
\textsc{FuseChat} comprises two main stages. Firstly, we undertake knowledge
fusion for structurally and scale-varied source LLMs to derive multiple target
LLMs of identical structure and size via lightweight fine-tuning. Then, these
target LLMs are merged within the parameter space, wherein we propose a novel
method for determining the merging weights based on the variation ratio of
parameter matrices before and after fine-tuning. We validate our approach using
three prominent chat LLMs with diverse architectures and scales, namely
\texttt{NH2-Mixtral-8x7B}, \texttt{NH2-Solar-10.7B}, and
\texttt{OpenChat-3.5-7B}. Experimental results spanning various chat domains
demonstrate the superiority of \texttt{\textsc{FuseChat}-7B} across a broad
spectrum of chat LLMs at 7B and 34B scales, even surpassing \texttt{GPT-3.5
(March)} and approaching \texttt{Mixtral-8x7B-Instruct}. Our code, model
weights, and data are openly accessible at
https://github.com/fanqiwan/FuseLLM.

通过知识融合技术将结构和规模不同的源语言模型（LLM）融合为具有相同结构和大小的目标 LLM，从而实现对聊天 LLM 的融合，该方法在聊天领域中的实验结果表明其在多个尺度上优于其他聊天 LLM 模型，并可达到类似于 Mixtral-8x7B-Instruct 的性能水平。