The scarcity of non-English data limits the development of non-English large
language models (LLMs). Transforming English-centric LLMs to non-English has
been identified as an effective and resource-efficient method. Previous works
start from base LLMs and perform knowledge distillation (KD) with data
generated by stronger LLMs, e.g. GPT-4. Compared to base LLMs, chat LLMs are
further optimized for advanced abilities, e.g. multi-turn conversation and
human preference alignment, and thus more powerful in both helpfulness and
safety. However, transforming a chat LLM involves two critical issues: (1) How
can we effectively transfer advanced abilities without their supervised data?
(2) How can we prevent the original knowledge from catastrophic forgetting
during transformation? We target these issues by introducing a simple framework
called TransLLM. For the first issue, TransLLM divides the transfer problem
into some common sub-tasks with the translation chain-of-thought, which uses
the translation as the bridge between English and non-English step-by-step. We
further enhance the performance of sub-tasks with publicly available data. For
the second issue, we propose a method comprising two synergistic components:
low-rank adaptation for training to maintain the original LLM parameters, and
recovery KD, which utilizes data generated by the chat LLM itself to recover
the original knowledge from the frozen parameters. In the experiments, we
transform the LLaMA-2-chat-7B to the Thai language. Our method, using only
single-turn data, outperforms strong baselines and ChatGPT on multi-turn
benchmark MT-bench. Furthermore, our method, without safety data, rejects more
harmful queries of safety benchmark AdvBench than both ChatGPT and GPT-4.

非英语大型语言模型（LLM）的稀缺性限制了其发展。将英语中心的 LLM 转化为非英语被认为是一种有效且资源高效的方法。我们介绍了一个名为 TransLLM 的简单框架，针对聊天型 LLM 转化问题提出了两个关键问题并给出解决方案。通过使用翻译作为连接英语和非英语的桥梁，TransLLM 将转换问题细分为多个常见的子任务。我们通过公开可用的数据进一步提高子任务的性能。在实验中，我们将 LLaMA-2-chat-7B 转化为泰语，仅使用单轮数据的方法，在多轮基准测试 MT-bench 上优于强基准和 ChatGPT。此外，没有安全数据的情况下，我们的方法在安全基准测试 AdvBench 上拒绝了比 ChatGPT 和 GPT-4 更多的有害查询。

为什么不将聊天型大语言模型转换为非英语？

Why Not Transform Chat Large Language Models to Non-English?

While training large language models (LLMs) from scratch can indeed lead to
models with distinct capabilities and strengths, this approach incurs
substantial costs and may lead to potential redundancy in competencies. An
alternative strategy is to combine existing LLMs into a more robust LLM,
thereby diminishing the necessity for expensive pre-training. However, due to
the diverse architectures of LLMs, direct parameter blending proves to be
unfeasible. Recently, \textsc{FuseLLM} introduced the concept of knowledge
fusion to transfer the collective knowledge of multiple structurally varied
LLMs into a target LLM through lightweight continual training. In this report,
we extend the scalability and flexibility of the \textsc{FuseLLM} framework to
realize the fusion of chat LLMs, resulting in \textsc{FuseChat}.
\textsc{FuseChat} comprises two main stages. Firstly, we undertake knowledge
fusion for structurally and scale-varied source LLMs to derive multiple target
LLMs of identical structure and size via lightweight fine-tuning. Then, these
target LLMs are merged within the parameter space, wherein we propose a novel
method for determining the merging weights based on the variation ratio of
parameter matrices before and after fine-tuning. We validate our approach using
three prominent chat LLMs with diverse architectures and scales, namely
\texttt{NH2-Mixtral-8x7B}, \texttt{NH2-Solar-10.7B}, and
\texttt{OpenChat-3.5-7B}. Experimental results spanning various chat domains
demonstrate the superiority of \texttt{\textsc{FuseChat}-7B} across a broad
spectrum of chat LLMs at 7B and 34B scales, even surpassing \texttt{GPT-3.5
(March)} and approaching \texttt{Mixtral-8x7B-Instruct}. Our code, model
weights, and data are openly accessible at
https://github.com/fanqiwan/FuseLLM.

通过知识融合技术将结构和规模不同的源语言模型（LLM）融合为具有相同结构和大小的目标 LLM，从而实现对聊天 LLM 的融合，该方法在聊天领域中的实验结果表明其在多个尺度上优于其他聊天 LLM 模型，并可达到类似于 Mixtral-8x7B-Instruct 的性能水平。