While understanding and removing gender biases in language models has been a long-standing problem in Natural Language Processing, prior research work has primarily been limited to English. In this work, we investigate some of the challenges with evaluating and mitigating biases in multilingual settings which stem from a lack of existing benchmarks and resources for bias evaluation beyond English especially for non-western context. In this paper, we first create a benchmark for evaluating gender biases in pre-trained masked language models by extending DisCo to different Indian languages using human annotations. We extend various debiasing methods to work beyond English and evaluate their effectiveness for SOTA massively multilingual models on our proposed metric. Overall, our work highlights the challenges that arise while studying social biases in multilingual settings and provides resources as well as mitigation techniques to take a step toward scaling to more languages.

这项研究探讨在多语言环境中评估和减少性别偏见在语言模型中的挑战，并通过 DisCo 扩展到不同的印度语言来创建了一个评估预训练屏蔽语言模型中性别偏见的基准，同时评估了各种方法对 SOTA 大规模多语言模型减轻此类偏见的有效性。

评估和减轻多语言环境中的性别偏见