Large language models (LLMs) have shown complementary strengths in various
tasks and instances, motivating the research of ensembling LLMs to push the
frontier leveraging the wisdom of the crowd. Existing work achieves this
objective via training the extra reward model or fusion model to select or fuse
all candidate answers. However, these methods pose a great challenge to the
generalizability of the trained models. Besides, existing methods use the
textual responses as communication media, ignoring the rich information in the
inner representations of neural networks. Therefore, we propose a training-free
ensemble framework DEEPEN, averaging the probability distributions outputted by
different LLMs. A key challenge in this paradigm is the vocabulary discrepancy
between heterogeneous LLMs, which hinders the operation of probability
distribution averaging. To address this challenge, DEEPEN maps the probability
distribution of each model from the probability space to a universe relative
space based on the relative representation theory, and performs aggregation.
Then, the result of aggregation is mapped back to the probability space of one
LLM via a search-based inverse transformation to determine the generated token.
We conduct experiments on the ensemble of various LLMs of 6B to 70B.
Experimental results show that DEEPEN achieves consistent improvements across
six popular benchmarks involving subject examination, reasoning and
knowledge-QA, proving the effectiveness of our approach.

我们提出了一个训练免费的集成框架 DEEPEN，通过平均不同大型语言模型输出的概率分布来解决现有方法中的词汇差异问题，并在涉及主观考试、推理和知识问答的六个流行基准测试中获得了一致的改进效果。