Regulatory efforts to protect against algorithmic bias have taken on
increased urgency with rapid advances in large language models (LLMs), which
are machine learning models that can achieve performance rivaling human experts
on a wide array of tasks. A key theme of these initiatives is algorithmic
"auditing," but current regulations -- as well as the scientific literature --
provide little guidance on how to conduct these assessments. Here we propose
and investigate one approach for auditing algorithms: correspondence
experiments, a widely applied tool for detecting bias in human judgements. In
the employment context, correspondence experiments aim to measure the extent to
which race and gender impact decisions by experimentally manipulating elements
of submitted application materials that suggest an applicant's demographic
traits, such as their listed name. We apply this method to audit candidate
assessments produced by several state-of-the-art LLMs, using a novel corpus of
applications to K-12 teaching positions in a large public school district. We
find evidence of moderate race and gender disparities, a pattern largely robust
to varying the types of application material input to the models, as well as
the framing of the task to the LLMs. We conclude by discussing some important
limitations of correspondence experiments for auditing algorithms.

在大型语言模型中检测算法偏见的对应实验方法，发现适用于 K-12 教职申请的各种模型存在中等程度的种族和性别差异，但同时也存在一些重要的限制。