Recent work has considered whether large language models (LLMs) can function
as planners: given a task, generate a plan. We investigate whether LLMs can
serve as generalized planners: given a domain and training tasks, generate a
program that efficiently produces plans for other tasks in the domain. In
particular, we consider PDDL domains and use GPT-4 to synthesize Python
programs. We also consider (1) Chain-of-Thought (CoT) summarization, where the
LLM is prompted to summarize the domain and propose a strategy in words before
synthesizing the program; and (2) automated debugging, where the program is
validated with respect to the training tasks, and in case of errors, the LLM is
re-prompted with four types of feedback. We evaluate this approach in seven
PDDL domains and compare it to four ablations and four baselines. Overall, we
find that GPT-4 is a surprisingly powerful generalized planner. We also
conclude that automated debugging is very important, that CoT summarization has
non-uniform impact, that GPT-4 is far superior to GPT-3.5, and that just two
training tasks are often sufficient for strong generalization.

本研究考察了大型语言模型是否可作为广义规划器，提供域和训练任务后，生成一个能够有效产生其他任务规划的程序。实验使用 PDDL 领域和 GPT-4 来合成 Python 程序，并考虑了 CoT 摘要和自动调试。研究发现，GPT-4 是一个出乎意料的功能强大的广义规划器，且自动调试非常重要，而 CoT 摘要影响不均。

使用预训练大型语言模型进行 PDDL 领域的广义规划

Generalized Planning in PDDL Domains with Pretrained Large Language  Models

The joint task of bug localization and program repair is an integral part of
the software development process. In this work we present DeepDebug, an
approach to automated debugging using large, pretrained transformers. We begin
by training a bug-creation model on reversed commit data for the purpose of
generating synthetic bugs. We apply these synthetic bugs toward two ends.
First, we directly train a backtranslation model on all functions from 200K
repositories. Next, we focus on 10K repositories for which we can execute
tests, and create buggy versions of all functions in those repositories that
are covered by passing tests. This provides us with rich debugging information
such as stack traces and print statements, which we use to finetune our model
which was pretrained on raw source code. Finally, we strengthen all our models
by expanding the context window beyond the buggy function itself, and adding a
skeleton consisting of that function's parent class, imports, signatures,
docstrings, and method bodies, in order of priority. On the QuixBugs benchmark,
we increase the total number of fixes found by over 50%, while also decreasing
the false positive rate from 35% to 5% and decreasing the timeout from six
hours to one minute. On our own benchmark of executable tests, our model fixes
68% of all bugs on its first attempt without using traces, and after adding
traces it fixes 75% on first attempt. We will open-source our framework and
validation set for evaluating on executable tests.

DeepDebug 是一种使用大型预训练变压器的自动调试方法，可通过训练反转的提交数据来生成人工合成缺陷，并通过将这些缺陷应用于功能测试，提供调试信息并改进模型的预训练，并在 QuixBugs 测试中取得良好成果。

DeepDebug: 使用堆栈跟踪、回译和代码框架修复 Python 错误

DeepDebug: Fixing Python Bugs Using Stack Traces, Backtranslation, and  Code Skeletons

Software debugging has been shown to utilize upwards of half of developers'
time. Yet, machine programming (MP), the field concerned with the automation of
software (and hardware) development, has recently made strides in both research
and production-quality automated debugging systems. In this paper we present
ControlFlag, a self-supervised MP system that aims to improve debugging by
attempting to detect idiosyncratic pattern violations in software control
structures. ControlFlag also suggests possible corrections in the event an
anomalous pattern is detected. We present ControlFlag's design and provide an
experimental evaluation and analysis of its efficacy in identifying potential
programming errors in production-quality software. As a first concrete evidence
towards improving software quality, ControlFlag has already found an anomaly in
CURL that has been acknowledged and fixed by its developers. We also discuss
future extensions of ControlFlag.

ControlFlag 是一种自主学习的机器编程系统，通过检测软件控制结构中的特殊模式违规来改善调试，发现出 CURL 存在的问题并提供了改正建议。