While humans sometimes do show the capability of correcting their own
erroneous guesses with self-critiquing, there seems to be no basis for that
assumption in the case of LLMs.

人类具备自我批评能力，但似乎在语言模型的情况下缺乏基础。

大型语言模型是否能够推理和规划？

Can Large Language Models Reason and Plan?

There have been widespread claims about Large Language Models (LLMs) being
able to successfully verify or self-critique their candidate solutions in
reasoning problems in an iterative mode. Intrigued by those claims, in this
paper we set out to investigate the verification/self-critiquing abilities of
large language models in the context of planning. We evaluate a planning system
that employs LLMs for both plan generation and verification. We assess the
verifier LLM's performance against ground-truth verification, the impact of
self-critiquing on plan generation, and the influence of varying feedback
levels on system performance. Using GPT-4, a state-of-the-art LLM, for both
generation and verification, our findings reveal that self-critiquing appears
to diminish plan generation performance, especially when compared to systems
with external, sound verifiers and the LLM verifiers in that system produce a
notable number of false positives, compromising the system's reliability.
Additionally, the nature of feedback, whether binary or detailed, showed
minimal impact on plan generation. Collectively, our results cast doubt on the
effectiveness of LLMs in a self-critiquing, iterative framework for planning
tasks.

几乎不存在自我评估的大型语言模型能够成功地在迭代模式下验证或自我批评其候选解决方案的推理问题中。该研究评估了一种利用大型语言模型进行计划生成和验证的计划系统，发现自我批评似乎削弱了计划生成的性能，并且系统的可靠性受到 LLM 验证器产生的显著错误结果的影响，二进制或详细反馈对计划生成几乎没有影响。总体而言，这些结果对于 LLMs 在计划任务的自我批评迭代框架中的有效性产生了质疑。