The increasing popularity of AI, particularly Large Language Models (LLMs),
has significantly impacted various domains, including Software Engineering.
This study explores the integration of AI tools in software engineering
practices within a large organization. We focus on ANZ Bank, which employs over
5000 engineers covering all aspects of the software development life cycle.
This paper details an experiment conducted using GitHub Copilot, a notable AI
tool, within a controlled environment to evaluate its effectiveness in
real-world engineering tasks. Additionally, this paper shares initial findings
on the productivity improvements observed after GitHub Copilot was adopted on a
large scale, with about 1000 engineers using it. ANZ Bank's six-week experiment
with GitHub Copilot included two weeks of preparation and four weeks of active
testing. The study evaluated participant sentiment and the tool's impact on
productivity, code quality, and security. Initially, participants used GitHub
Copilot for proposed use-cases, with their feedback gathered through regular
surveys. In the second phase, they were divided into Control and Copilot
groups, each tackling the same Python challenges, and their experiences were
again surveyed. Results showed a notable boost in productivity and code quality
with GitHub Copilot, though its impact on code security remained inconclusive.
Participant responses were overall positive, confirming GitHub Copilot's
effectiveness in large-scale software engineering environments. Early data from
1000 engineers also indicated a significant increase in productivity and job
satisfaction.

在 ANZ 银行的软件工程实践中，探索了将人工智能工具集成到软件工程中的方法。通过使用 GitHub Copilot 进行实验，并在大规模使用中观察了其对生产力、代码质量以及安全性的影响，结果表明 GitHub Copilot 在大规模软件工程环境中非常有效。