Distributional word vectors have recently been shown to encode many of the
human biases, most notably gender and racial biases, and models for attenuating
such biases have consequently been proposed. However, existing models and
studies (1) operate on under-specified and mutually differing bias definitions,
(2) are tailored for a particular bias (e.g., gender bias) and (3) have been
evaluated inconsistently and non-rigorously. In this work, we introduce a
general framework for debiasing word embeddings. We operationalize the
definition of a bias by discerning two types of bias specification: explicit
and implicit. We then propose three debiasing models that operate on explicit
or implicit bias specifications and that can be composed towards more robust
debiasing. Finally, we devise a full-fledged evaluation framework in which we
couple existing bias metrics with newly proposed ones. Experimental findings
across three embedding methods suggest that the proposed debiasing models are
robust and widely applicable: they often completely remove the bias both
implicitly and explicitly without degradation of semantic information encoded
in any of the input distributional spaces. Moreover, we successfully transfer
debiasing models, by means of cross-lingual embedding spaces, and remove or
attenuate biases in distributional word vector spaces of languages that lack
readily available bias specifications.

本论文提出了一个通用的去除词向量中性别、种族等偏见的方法，并通过新的评估指标验证了该框架的有效性，同时在跨语言文本中成功实现了去偏。

分布式词向量空间中隐式和显式去偏见的一般框架

A General Framework for Implicit and Explicit Debiasing of  Distributional Word Vector Spaces

Semantic specialization is the process of fine-tuning pre-trained
distributional word vectors using external lexical knowledge (e.g., WordNet) to
accentuate a particular semantic relation in the specialized vector space.
While post-processing specialization methods are applicable to arbitrary
distributional vectors, they are limited to updating only the vectors of words
occurring in external lexicons (i.e., seen words), leaving the vectors of all
other words unchanged. We propose a novel approach to specializing the full
distributional vocabulary. Our adversarial post-specialization method
propagates the external lexical knowledge to the full distributional space. We
exploit words seen in the resources as training examples for learning a global
specialization function. This function is learned by combining a standard
L2-distance loss with an adversarial loss: the adversarial component produces
more realistic output vectors. We show the effectiveness and robustness of the
proposed method across three languages and on three tasks: word similarity,
dialog state tracking, and lexical simplification. We report consistent
improvements over distributional word vectors and vectors specialized by other
state-of-the-art specialization frameworks. Finally, we also propose a
cross-lingual transfer method for zero-shot specialization which successfully
specializes a full target distributional space without any lexical knowledge in
the target language and without any bilingual data.

本文提出了一种基于对抗训练的后加工特化方法，通过整合外部词汇知识到整个分布式语义空间中，利用资源中出现的词作为训练样例，结合标准 L2 距离和对抗损失，学习全局特化函数，表现良好并提高了词相似度、对话状态跟踪和词汇简化等任务的效果，并提出了一种零射击专业化的跨语言转移方法。