Large Language Models have many methods for solving the same problem. This
introduces novel strengths (different methods may work well for different
problems) and weaknesses (it may be difficult for users to know which method to
use). In this paper, we introduce Multi-Method Self-Training (MMST), where one
method is trained on the filtered outputs of another, allowing us to augment
the strengths and ameliorate the weaknesses of each method. Using a 176B
parameter model trained on both language and code, we show that MMST can 1)
improve the less performant method (up to 30%) making the model easier to use,
2) improve the more performant method (up to 32.2%) making the model more
performant, and 3) improve the performance of related but distinct tasks (up to
10.3%) by improving the ability of the model to generate rationales. We then
conduct ablation analyses to explore why MMST works. We show that MMST
generates more data than traditional self-training, but the improvement in
performance is driven by the use of multiple methods. We also analyze
prompt-engineering and anti-correlated performance between methods as means of
making MMST more effective. We hope the evidence from our paper motivates
machine learning researchers to explore ways in which advances in language
models allow for new forms of training.

大型语言模型通过多种方法解决问题，在这篇论文中，我们介绍了多方法自训练（MMST），其中一种方法是在另一种方法的过滤输出上训练，从而增强每种方法的优势并改善它们的缺点。通过对同时训练了语言和代码的含有 176B 参数的模型，我们展示了 MMST 可以提升性能较差的方法（最高可达 30%），使模型更易于使用，改善性能较好的方法（最高可达 32.2%），使模型表现更好，并通过提高模型生成原理的能力，改善相关但不同的任务的性能（最高可达 10.3%）。然后，我们进行了消融分析来探索 MMST 的工作原理，发现 MMST 产生的数据比传统的自训练更多，但性能的提升是由多种方法的使用驱动的。我们还分析了提示工程和方法之间的反相关性能，以提高 MMST 的有效性。我们希望我们论文中的证据能够激励机器学习研究人员探索语言模型进步带来的新型训练方法。