Reasoning capabilities are crucial for Large Language Models (LLMs), yet a
notable gap exists between English and non-English languages. To bridge this
disparity, some works fine-tune LLMs to relearn reasoning capabilities in
non-English languages, while others replace non-English inputs with an external
model's outputs such as English translation text to circumvent the challenge of
LLM understanding non-English. Unfortunately, these methods often underutilize
the built-in skilled reasoning and useful language understanding capabilities
of LLMs. In order to better utilize the minds of reasoning and language
understanding in LLMs, we propose a new method, namely MindMerger, which merges
LLMs with the external language understanding capabilities from multilingual
models to boost the multilingual reasoning performance. Furthermore, a two-step
training scheme is introduced to first train to embeded the external
capabilities into LLMs and then train the collaborative utilization of the
external capabilities and the built-in capabilities in LLMs. Experiments on
three multilingual reasoning datasets and a language understanding dataset
demonstrate that MindMerger consistently outperforms all baselines, especially
in low-resource languages. Without updating the parameters of LLMs, the average
accuracy improved by 6.7% and 8.0% across all languages and low-resource
languages on the MGSM dataset, respectively.

LLMs 与多语言模型中的外部语言理解能力的融合方法 MindMerger 可提高多语言推理性能。在多个语言推理数据集上的实验证明，MindMerger 相较于其他方法具有更好的性能，尤其对于资源稀缺的语言。在 MGSM 数据集上，不更新 LLMs 的参数，平均准确率分别提升了 6.7% 和 8.0%，涵盖所有语言和资源稀缺的语言。