Leveraging inference-time search in large language models has proven effective in further enhancing a trained model's capability to solve complex mathematical and reasoning problems. However, this approach significantly increases computational costs and inference time, as the model must generate and evaluate multiple candidate solutions to identify a viable reasoning path. To address this, we propose an effective approach that integrates search capabilities directly into the model by fine-tuning it using both successful (learning) and failed reasoning paths (forgetting) derived from diverse search methods. While fine-tuning the model with these data might seem straightforward, we identify a critical issue: the model's search capability tends to degrade rapidly if fine-tuning is performed naively. We show that this degradation can be substantially mitigated by employing a smaller learning rate. Extensive experiments on the challenging Game-of-24 and Countdown mathematical reasoning benchmarks show that our approach not only outperforms both standard fine-tuning and inference-time search baselines but also significantly reduces inference time by 180$\times$.

本研究针对大型语言模型在解决复杂数学和推理问题时的推理能力不足这一问题，提出了一种新的方法，通过结合成功和失败的推理路径进行微调，直接增强模型的搜索能力。研究发现，采用较小的学习率可以有效减缓模型的性能下降，从而在多个推理基准测试中，显著提高了模型的表现并减少了180倍的推理时间。

通过学习与遗忘教导大型语言模型推理