Sense embedding learning methods learn different embeddings for the different
senses of an ambiguous word. One sense of an ambiguous word might be socially
biased while its other senses remain unbiased. In comparison to the numerous
prior work evaluating the social biases in pretrained word embeddings, the
biases in sense embeddings have been relatively understudied. We create a
benchmark dataset for evaluating the social biases in sense embeddings and
propose novel sense-specific bias evaluation measures. We conduct an extensive
evaluation of multiple static and contextualised sense embeddings for various
types of social biases using the proposed measures. Our experimental results
show that even in cases where no biases are found at word-level, there still
exist worrying levels of social biases at sense-level, which are often ignored
by the word-level bias evaluation measures.

本文提出一种评估感觉嵌入社会偏见的基准数据集和新的感知特定偏见评估指标，并对多个静态和上下文感知嵌入进行广泛评估，发现即使在单词级别没有发现偏见的情况下，仍存在令人担忧的感知级别的社会偏见，这通常被单词级别偏见评估措施所忽略。