How can we harness the collective capabilities of multiple Large Language Models (LLMs) to create an even more powerful model? This question forms the foundation of our research, where we propose an innovative approach to weak-to-strong (w2s) generalization-a critical problem in AI alignment. Our work introduces an easy-to-hard (e2h) framework for studying the feasibility of w2s generalization, where weak models trained on simpler tasks collaboratively supervise stronger models on more complex tasks. This setup mirrors real-world challenges, where direct human supervision is limited. To achieve this, we develop a novel AdaBoost-inspired ensemble method, demonstrating that an ensemble of weak supervisors can enhance the performance of stronger LLMs across classification and generative tasks on difficult QA datasets. In several cases, our ensemble approach matches the performance of models trained on ground-truth data, establishing a new benchmark for w2s generalization. We observe an improvement of up to 14% over existing baselines and average improvements of 5% and 4% for binary classification and generative tasks, respectively. This research points to a promising direction for enhancing AI through collective supervision, especially in scenarios where labeled data is sparse or insufficient.

本研究聚焦于如何提升多个大规模语言模型（LLMs）的集体能力，以创建更强的模型，解决了人工智能对齐中的弱到强（w2s）泛化问题。我们提出了一种基于AdaBoost的集成方法，通过不同时任务上训练的弱模型指导在复杂任务上表现更强的模型，实验结果显示该方法在分类和生成任务上均超越现有基准，提升幅度高达14%。

EnsemW2S：能否利用大规模语言模型的集成来获得更强大的语言模型？