Process-level Reward Models (PRMs) are crucial for complex reasoning and decision-making tasks, where each intermediate step plays an important role in the reasoning process. Since language models are prone to various types of errors during the reasoning process, PRMs are required to possess nuanced capabilities for detecting various implicit error types in real-world scenarios. However, current benchmarks primarily focus on step correctness, failing to evaluate PRMs' performance systematically. To address this gap, we introduce PRMBench, a process-level benchmark specifically designed to assess the fine-grained error detection capabilities of PRMs. PRMBench comprises 6,216 carefully designed problems and 83,456 step-level labels, evaluating models across multiple dimensions, including simplicity, soundness, and sensitivity. In our experiments on 15 models, spanning both open-source PRMs and closed-source large language models prompted as critic models, we uncover significant weaknesses in current PRMs. These findings underscore the challenges inherent in process-level evaluation and highlight key directions for future research. We hope PRMBench can be a robust bench for advancing research on PRM evaluation and development.

本研究针对现有的过程级奖励模型（PRMs）评估不足的问题，提出了PRMBench基准，旨在系统性地评估PRMs在细粒度错误检测能力上的表现。通过设计6216个问题和83456个步骤级标签，研究揭示了当前PRMs的重大缺陷，强调了过程级评估的挑战，并为未来研究指明了方向。

PRMBench：一个细粒度且具有挑战性的过程级奖励模型基准