GitHub issue resolving recently has attracted significant attention from
academia and industry. SWE-bench is proposed to measure the performance in
resolving issues. In this paper, we propose CodeR, which adopts a multi-agent
framework and pre-defined task graphs to Repair & Resolve reported bugs and add
new features within code Repository. On SWE-bench lite, CodeR is able to solve
28.00% of issues, in the case of submitting only once for each issue. We
examine the performance impact of each design of CodeR and offer insights to
advance this research direction.

采用多智能体框架和预定义的任务图，提出了 CodeR，用于修复和解决代码存储库中报告的错误和添加新功能。在 SWE-bench lite 中，CodeR 能够解决 28.00％的问题，每个问题仅需提交一次。检验了 CodeR 的每个设计对性能的影响，并提供了推进这个研究方向的见解。

代码 R：多智能体和任务图的问题解决

CodeR: Issue Resolving with Multi-Agent and Task Graphs

Language models have outpaced our ability to evaluate them effectively, but
for their future development it is essential to study the frontier of their
capabilities. We consider real-world software engineering to be a rich,
sustainable, and challenging testbed for evaluating the next generation of
language models. We therefore introduce SWE-bench, an evaluation framework
including $2,294$ software engineering problems drawn from real GitHub issues
and corresponding pull requests across $12$ popular Python repositories. Given
a codebase along with a description of an issue to be resolved, a language
model is tasked with editing the codebase to address the issue. Resolving
issues in SWE-bench frequently requires understanding and coordinating changes
across multiple functions, classes, and even files simultaneously, calling for
models to interact with execution environments, process extremely long contexts
and perform complex reasoning that goes far beyond traditional code generation.
Our evaluations show that both state-of-the-art proprietary models and our
fine-tuned model SWE-Llama can resolve only the simplest issues. Claude 2 and
GPT-4 solve a mere $4.8$% and $1.7$% of instances respectively, even when
provided with an oracle retriever. Advances on SWE-bench represent steps
towards LMs that are more practical, intelligent, and autonomous.

评估了新一代语言模型在软件工程领域的能力，通过 SWE-bench 框架提供了包括软件工程问题和对应 Python 库的真实 GitHub 问题的解决方案，结果表明目前的模型在解决问题时仍存在困难，呼吁进一步提高模型在实际应用中的智能和自主能力。