Training AI models that generalize across tasks and domains has long been
among the open problems driving AI research. The emergence of Foundation Models
made it easier to obtain expert models for a given task, but the heterogeneity
of data that may be encountered at test time often means that any single expert
is insufficient. We consider the Fusion of Experts (FoE) problem of fusing
outputs of expert models with complementary knowledge of the data distribution
and formulate it as an instance of supervised learning. Our method is
applicable to both discriminative and generative tasks and leads to significant
performance improvements in image and text classification, text summarization,
multiple-choice QA, and automatic evaluation of generated text. We also extend
our method to the "frugal" setting where it is desired to reduce the number of
expert model evaluations at test time.

训练具有跨任务和领域泛化能力的 AI 模型一直以来都是推动 AI 研究的一个难题。本文提出了专家模型融合问题，并将其作为一种监督学习实例来解决。我们的方法适用于判别和生成任务，并在图像和文本分类、文本摘要、多项选择问答和生成文本的自动评估等方面取得了显著的性能提升。同时，我们还将方法扩展到了 “节约” 的设置中，以期在测试时减少专家模型的评估次数。