Large Language Models (LLMs) such as GPT-3.5, Bard, and Claude exhibit
applicability across numerous tasks. One domain of interest is their use in
algorithmic hiring, specifically in matching resumes with job categories. Yet,
this introduces issues of bias on protected attributes like gender, race and
maternity status. The seminal work of Bertrand & Mullainathan (2003) set the
gold-standard for identifying hiring bias via field experiments where the
response rate for identical resumes that differ only in protected attributes,
e.g., racially suggestive names such as Emily or Lakisha, is compared. We
replicate this experiment on state-of-art LLMs (GPT-3.5, Bard, Claude and
Llama) to evaluate bias (or lack thereof) on gender, race, maternity status,
pregnancy status, and political affiliation. We evaluate LLMs on two tasks: (1)
matching resumes to job categories; and (2) summarizing resumes with employment
relevant information. Overall, LLMs are robust across race and gender. They
differ in their performance on pregnancy status and political affiliation. We
use contrastive input decoding on open-source LLMs to uncover potential sources
of bias.

大型语言模型在算法招聘中的使用引发了与受保护属性（如性别、种族和孕期状态）的偏见问题。本研究基于开源的大型语言模型，在匹配简历与职位类别以及总结简历的两个任务上评估性别、种族、孕期状态和政治倾向的偏见。结果显示大型语言模型在种族和性别方面具有鲁棒性，但在孕期状态和政治倾向方面表现不同，通过对比输入解码方法揭示了潜在的偏见源。

探究 ChatGPT 时代算法招聘偏见：Emilly、Greg 是否比 Lakisha、Jamal 更具就业能力？

Are Emily and Greg Still More Employable than Lakisha and Jamal?  Investigating Algorithmic Hiring Bias in the Era of ChatGPT

Ensuring that large language models (LMs) are fair, robust and useful
requires an understanding of how different modifications to their inputs impact
the model's behaviour. In the context of open-text generation tasks, however,
such an evaluation is not trivial. For example, when introducing a model with
an input text and a perturbed, "contrastive" version of it, meaningful
differences in the next-token predictions may not be revealed with standard
decoding strategies. With this motivation in mind, we propose Contrastive Input
Decoding (CID): a decoding algorithm to generate text given two inputs, where
the generated text is likely given one input but unlikely given the other. In
this way, the contrastive generations can highlight potentially subtle
differences in how the LM output differs for the two inputs in a simple and
interpretable manner. We use CID to highlight context-specific biases that are
hard to detect with standard decoding strategies and quantify the effect of
different input perturbations.

提出对比性输入解码算法 (CID) 来量化大型语言模型对不同输入的行为变化，发现标准解码策略未能检测到的特定语境偏差。