Large language models (LLMs) have demonstrated remarkable capabilities across
a wide range of tasks. However, even the most advanced open-source LLMs, such
as the LLaMA family models, still face challenges when it comes to accurately
solving complex multi-step mathematical problems. In this paper, we present an
innovative process-oriented math verifier called \textbf{Math-Shepherd}, which
assigns a reward score to each step of the LLM's outputs on math problems. The
training of Math-Shepherd is achieved using automatically constructed
process-wise supervision data, breaking the bottleneck of heavy reliance on
manual annotation in existing work. With the guidance of Math-Shepherd, a
series of open-source LLMs demonstrate exceptional performance. Among them,
DeepSeek 67B \citep{DeepSeek-llm} stands out by achieving accuracy rates of
93.3\% on the GSM8K dataset and 48.1\% on the MATH dataset, without external
enhancement such as tool usage. Our Math-Shepherd also outperforms the
self-consistency method and other existing verification models. We believe that
automatic process supervision holds significant potential for the future
evolution of LLMs.

Math-Shepherd 是一种创新的过程导向数学验证器，使用自动生成的过程级监督数据进行训练，通过对 LLM 的输出逐步赋予奖励分数来解决复杂多步骤数学问题，指导一系列开源 LLMs 取得了非凡的性能，并在自洽性方法和其他验证模型方面表现优异，我们相信自动过程监督对 LLM 的未来演进具有重要潜力。