LLM-based assistants, such as GitHub Copilot and ChatGPT, have the potential
to generate code that fulfills a programming task described in a natural
language description, referred to as a prompt. The widespread accessibility of
these assistants enables users with diverse backgrounds to generate code and
integrate it into software projects. However, studies show that code generated
by LLMs is prone to bugs and may miss various corner cases in task
specifications. Presenting such buggy code to users can impact their
reliability and trust in LLM-based assistants. Moreover, significant efforts
are required by the user to detect and repair any bug present in the code,
especially if no test cases are available. In this study, we propose a
self-refinement method aimed at improving the reliability of code generated by
LLMs by minimizing the number of bugs before execution, without human
intervention, and in the absence of test cases. Our approach is based on
targeted Verification Questions (VQs) to identify potential bugs within the
initial code. These VQs target various nodes within the Abstract Syntax Tree
(AST) of the initial code, which have the potential to trigger specific types
of bug patterns commonly found in LLM-generated code. Finally, our method
attempts to repair these potential bugs by re-prompting the LLM with the
targeted VQs and the initial code. Our evaluation, based on programming tasks
in the CoderEval dataset, demonstrates that our proposed method outperforms
state-of-the-art methods by decreasing the number of targeted errors in the
code between 21% to 62% and improving the number of executable code instances
to 13%.

通过自我完善方法，在没有人工介入和测试用例的情况下，通过针对初始代码中的特定节点使用目标验证问题（VQs），提高 LLM 生成代码的可靠性，并通过针对特定的 VQs 和初始代码重新提示 LLM 来修复潜在的漏洞。评估结果表明，我们的方法优于现有方法，减少了代码中目标错误的数量达到 21% 至 62%，同时提高了可执行代码实例的数量达到 13%。