Modern code review is a process in which an incremental code contribution
made by a code author is reviewed by one or more peers before it is committed
to the version control system. An important element of modern code review is
verifying that code contributions adhere to best practices. While some of these
best practices can be automatically verified, verifying others is commonly left
to human reviewers. This paper reports on the development, deployment, and
evaluation of AutoCommenter, a system backed by a large language model that
automatically learns and enforces coding best practices. We implemented
AutoCommenter for four programming languages (C++, Java, Python, and Go) and
evaluated its performance and adoption in a large industrial setting. Our
evaluation shows that an end-to-end system for learning and enforcing coding
best practices is feasible and has a positive impact on the developer workflow.
Additionally, this paper reports on the challenges associated with deploying
such a system to tens of thousands of developers and the corresponding lessons
learned.

AutoCommenter 是一个自动学习和执行编码最佳实践的系统，针对四种编程语言进行了实现和评估，结果表明其对开发者工作流程有积极影响，并述及了在大规模部署中所面临的挑战和相应的经验教训。

现代代码审查中的人工智能辅助编码实践评估

AI-Assisted Assessment of Coding Practices in Modern Code Review

A great part of software development involves conceptualizing or
communicating the underlying procedures and logic that needs to be expressed in
programs. One major difficulty of programming is turning concept into code,
especially when dealing with the APIs of unfamiliar libraries. Recently, there
has been a proliferation of machine learning methods for code generation and
retrieval from natural language queries, but these have primarily been
evaluated purely based on retrieval accuracy or overlap of generated code with
developer-written code, and the actual effect of these methods on the developer
workflow is surprisingly unattested. We perform the first comprehensive
investigation of the promise and challenges of using such technology inside the
IDE, asking "at the current state of technology does it improve developer
productivity or accuracy, how does it affect the developer experience, and what
are the remaining gaps and challenges?" We first develop a plugin for the IDE
that implements a hybrid of code generation and code retrieval functionality,
and orchestrate virtual environments to enable collection of many user events.
We ask developers with various backgrounds to complete 14 Python programming
tasks ranging from basic file manipulation to machine learning or data
visualization, with or without the help of the plugin. While qualitative
surveys of developer experience are largely positive, quantitative results with
regards to increased productivity, code quality, or program correctness are
inconclusive. Analysis identifies several pain points that could improve the
effectiveness of future machine learning based code generation/retrieval
developer assistants, and demonstrates when developers prefer code generation
over code retrieval and vice versa. We release all data and software to pave
the road for future empirical studies and development of better models.

本文研究了在 IDE 中使用机器学习代码生成 / 检索技术对开发工作流程和体验产生的影响，设计了一款混合代码生成和检索的插件，并通过用户测试发现虽然开发者的体验得到了改善，但是对于提高生产率、代码质量和程序正确性等方面的具体效果并不确定，同时分析了需要改进的地方和开发者偏好。