Recent advances in large language models have shown capabilities that are
extraordinary and near-superhuman. These models operate with such complexity
that reliably evaluating and aligning them proves challenging for humans. This
leads to the natural question: can guidance from weak models (like humans)
adequately direct the capabilities of strong models? In a recent and somewhat
surprising work, Burns et al. (2023) empirically demonstrated that when strong
models (like GPT-4) are finetuned using labels generated by weak supervisors
(like GPT-2), the strong models outperform their weaker counterparts -- a
phenomenon they term weak-to-strong generalization.
In this work, we present a theoretical framework for understanding
weak-to-strong generalization. Specifically, we show that the improvement in
performance achieved by strong models over their weaker counterparts is
quantified by the misfit error incurred by the strong model on labels generated
by the weaker model. Our theory reveals several curious algorithmic insights.
For instance, we can predict the amount by which the strong model will improve
over the weak model, and also choose among different weak models to train the
strong model, based on its misfit error. We validate our theoretical findings
through various empirical assessments.

使用弱模型生成的标签对强模型进行微调可以显著提高强模型性能，本文提出了一个理论框架来解释这种弱到强的泛化现象，并通过多种实证评估验证了理论发现。