Scientific equation discovery is a fundamental task in the history of scientific progress, enabling the derivation of laws governing natural phenomena. Recently, Large Language Models (LLMs) have gained interest for this task due to their potential to leverage embedded scientific knowledge for hypothesis generation. However, evaluating the true discovery capabilities of these methods remains challenging, as existing benchmarks often rely on common equations that are susceptible to memorization by LLMs, leading to inflated performance metrics that do not reflect discovery. In this paper, we introduce LLM-SRBench, a comprehensive benchmark with 239 challenging problems across four scientific domains specifically designed to evaluate LLM-based scientific equation discovery methods while preventing trivial memorization. Our benchmark comprises two main categories: LSR-Transform, which transforms common physical models into less common mathematical representations to test reasoning beyond memorized forms, and LSR-Synth, which introduces synthetic, discovery-driven problems requiring data-driven reasoning. Through extensive evaluation of several state-of-the-art methods, using both open and closed LLMs, we find that the best-performing system so far achieves only 31.5% symbolic accuracy. These findings highlight the challenges of scientific equation discovery, positioning LLM-SRBench as a valuable resource for future research.

该研究针对科学方程发现的评估问题，提出了一个名为LLM-SRBench的新基准，旨在克服现有基准的局限性，减少对常见方程的记忆性依赖。该基准涵盖239个具有挑战性的问题，提供两种主要类别，以测试语言模型的推理能力和数据驱动的发现能力。研究表明，现有先进方法的最佳表现仅为31.5%的符号准确率，凸显了科学方程发现的复杂性。

LLM-SRBench：基于大型语言模型的科学方程发现新基准