Pre-trained language models (LMs) may perpetuate biases originating in their
training corpus to downstream models. We focus on artifacts associated with the
representation of given names (e.g., Donald), which, depending on the corpus,
may be associated with specific entities, as indicated by next token prediction
(e.g., Trump). While helpful in some contexts, grounding happens also in
under-specified or inappropriate contexts. For example, endings generated for
`Donald is a' substantially differ from those of other names, and often have
more-than-average negative sentiment. We demonstrate the potential effect on
downstream tasks with reading comprehension probes where name perturbation
changes the model answers. As a silver lining, our experiments suggest that
additional pre-training on different corpora may mitigate this bias.

本文针对预训练语言模型 (LMs) 可能在下游模型中保留其训练语料库中的偏差进行了研究，重点关注给定名称（例如 Donald）表示中出现的问题，可能会因不同的语料库与特定实体相关，如后续标记预测（例如 Trump），通过阅读理解探针演示了名称扰动对模型答案的潜在影响，我们的实验表明，对不同语料库进行额外的预训练可能会减轻这种偏差。