Recent work has considered whether large language models (LLMs) can function
as planners: given a task, generate a plan. We investigate whether LLMs can
serve as generalized planners: given a domain and training tasks, generate a
program that efficiently produces plans for other tasks in the domain. In
particular, we consider PDDL domains and use GPT-4 to synthesize Python
programs. We also consider (1) Chain-of-Thought (CoT) summarization, where the
LLM is prompted to summarize the domain and propose a strategy in words before
synthesizing the program; and (2) automated debugging, where the program is
validated with respect to the training tasks, and in case of errors, the LLM is
re-prompted with four types of feedback. We evaluate this approach in seven
PDDL domains and compare it to four ablations and four baselines. Overall, we
find that GPT-4 is a surprisingly powerful generalized planner. We also
conclude that automated debugging is very important, that CoT summarization has
non-uniform impact, that GPT-4 is far superior to GPT-3.5, and that just two
training tasks are often sufficient for strong generalization.

本研究考察了大型语言模型是否可作为广义规划器，提供域和训练任务后，生成一个能够有效产生其他任务规划的程序。实验使用 PDDL 领域和 GPT-4 来合成 Python 程序，并考虑了 CoT 摘要和自动调试。研究发现，GPT-4 是一个出乎意料的功能强大的广义规划器，且自动调试非常重要，而 CoT 摘要影响不均。