We investigate a failure mode that arises during the training of reasoning models, where the diversity of generations begins to collapse, leading to suboptimal test-time scaling. Notably, the Pass@1 rate reliably improves during supervised finetuning (SFT), but Pass@k rapidly deteriorates. Surprisingly, a simple intervention of interpolating the weights of the latest SFT checkpoint with an early checkpoint, otherwise known as WiSE-FT, almost completely recovers Pass@k while also improving Pass@1. The WiSE-FT variant achieves better test-time scaling (Best@k, majority vote) and achieves superior results with less data when tuned further by reinforcement learning. Finally, we find that WiSE-FT provides complementary performance gains that cannot be achieved only through diversity-inducing decoding strategies, like temperature scaling. We formalize a bias-variance tradeoff of Pass@k with respect to the expectation and variance of Pass@1 over the test distribution. We find that WiSE-FT can reduce bias and variance simultaneously, while temperature scaling inherently trades-off between bias and variance.

本研究解决了推理模型训练过程中的一种失效模式，即生成的多样性开始崩溃，导致测试时扩展性不佳。我们提出了一种简单的干预措施，即将最新的监督微调检查点与早期检查点的权重进行插值（WiSE-FT），几乎完全恢复了 Pass@k，并改善了 Pass@1，显示了在数据更少的情况下更优的结果和测试时扩展性。

权重集成改善语言模型的推理能力