We study the bias in several state-of-the-art named entity recognition (NER)
models---specifically, a difference in the ability to recognize male and female
names as PERSON entity types. We evaluate NER models on a dataset containing
139 years of U.S. census baby names and find that relatively more female names,
as opposed to male names, are not recognized as PERSON entities. We study the
extent of this bias in several NER systems that are used prominently in
industry and academia. In addition, we also report a bias in the datasets on
which these models were trained. The result of this analysis yields a new
benchmark for gender bias evaluation in named entity recognition systems. The
data and code for the application of this benchmark will be publicly available
for researchers to use.

该论文研究了现有几种命名实体识别模型的偏见，特别是在将男性和女性姓名识别为人名实体中存在的差异。该研究评估了包含 139 年美国人口普查婴儿姓名的数据集上的 NER 模型，并发现相对较多的女性姓名未被识别为人名实体。作者还研究了这种偏差在几个业界和学术界中广泛使用的 NER 系统中的程度，并报告了这些模型的训练数据集中存在的偏见。该分析的结果产生了一个新的用于命名实体识别系统中性别偏见评估的基准。该基准的数据和代码将公开提供给研究人员使用。