Recent advancements in code-fluent Large Language Models (LLMs) enabled the
research on repository-level code editing. In such tasks, the model navigates
and modifies the entire codebase of a project according to request. Hence, such
tasks require efficient context retrieval, i.e., navigating vast codebases to
gather relevant context. Despite the recognized importance of context
retrieval, existing studies tend to approach repository-level coding tasks in
an end-to-end manner, rendering the impact of individual components within
these complicated systems unclear. In this work, we decouple the task of
context retrieval from the other components of the repository-level code
editing pipelines. We lay the groundwork to define the strengths and weaknesses
of this component and the role that reasoning plays in it by conducting
experiments that focus solely on context retrieval. We conclude that while the
reasoning helps to improve the precision of the gathered context, it still
lacks the ability to identify its sufficiency. We also outline the ultimate
role of the specialized tools in the process of context gathering. The code
supplementing this paper is available at
this https URL

这项研究探讨了使用代码流畅的大型语言模型（LLMs）进行存储库级代码编辑的最新进展，着重解耦上下文检索的任务以评估其组件的优劣，并指出推理在上下文获取中提高准确性的作用，但仍无法确定其充分性及专用工具在该过程中的最终角色。

关于仓库级代码编辑中推理的重要性

On The Importance of Reasoning for Context Retrieval in Repository-Level  Code Editing

Large Language Models (LLMs) like GPT-4 have revolutionized natural language
processing, showing remarkable linguistic proficiency and reasoning
capabilities. However, their application in strategic multi-agent
decision-making environments is hampered by significant limitations including
poor mathematical reasoning, difficulty in following instructions, and a
tendency to generate incorrect information. These deficiencies hinder their
performance in strategic and interactive tasks that demand adherence to nuanced
game rules, long-term planning, exploration in unknown environments, and
anticipation of opponents' moves. To overcome these obstacles, this paper
presents a novel LLM agent framework equipped with memory and specialized tools
to enhance their strategic decision-making capabilities. We deploy the tools in
a number of economically important environments, in particular bilateral
bargaining and multi-agent and dynamic mechanism design. We employ quantitative
metrics to assess the framework's performance in various strategic
decision-making problems. Our findings establish that our enhanced framework
significantly improves the strategic decision-making capability of LLMs. While
we highlight the inherent limitations of current LLM models, we demonstrate the
improvements through targeted enhancements, suggesting a promising direction
for future developments in LLM applications for interactive environments.

本文针对目前语言模型在战略多代理决策环境中存在的局限性，提出了一种新颖的语言模型代理框架，通过加入记忆和专门设计的工具，提升了其战略决策能力。通过在多个重要经济环境中的应用，特别是双边协商和多代理动态机制设计，我们使用定量指标评估了该框架在各种战略决策问题中的表现。研究结果表明，我们改进后的框架显著提升了语言模型的战略决策能力，同时也强调了当前语言模型的固有局限性，为未来在交互环境中的应用指明了一个有前景的方向。