Ensembling multiple models has always been an effective approach to push the
limits of existing performance and is widely used in classification tasks by
simply averaging the classification probability vectors from multiple
classifiers to achieve better accuracy. However, in the thriving open-source
Large Language Model (LLM) community, ensembling methods are rare and typically
limited to ensembling the full-text outputs of LLMs, such as selecting the best
output using a ranker, which leads to underutilization of token-level
probability information. In this paper, we treat the Generation of each token
by LLMs as a Classification (GaC) for ensembling. This approach fully exploits
the probability information at each generation step and better prevents LLMs
from producing early incorrect tokens that lead to snowballing errors. In
experiments, we ensemble state-of-the-art LLMs on several benchmarks, including
exams, mathematics and reasoning, and observe that our method breaks the
existing community performance ceiling. Furthermore, we observed that most of
the tokens in the answer are simple and do not affect the correctness of the
final answer. Therefore, we also experimented with ensembling only key tokens,
and the results showed better performance with lower latency across benchmarks.

利用生成式语言模型（GaC），将多个大型语言模型（LLM）集成在一起，在多项基准测试中突破了现有的性能限制；此外，仅集成关键标记也表现出更好的性能和更低的延迟。