BERT and other large-scale language models (LMs) contain gender and racial bias. They also exhibit other dimensions of social bias, most of which have not been studied in depth, and some of which vary depending on the language. In this paper, we study ethnic bias and how it varies across languages by analyzing and mitigating ethnic bias in monolingual BERT for English, German, Spanish, Korean, Turkish, and Chinese. To observe and quantify ethnic bias, we develop a novel metric called Categorical Bias score. Then we propose two methods for mitigation; first using a multilingual model, and second using contextual word alignment of two monolingual models. We compare our proposed methods with monolingual BERT and show that these methods effectively alleviate the ethnic bias. Which of the two methods works better depends on the amount of NLP resources available for that language. We additionally experiment with Arabic and Greek to verify that our proposed methods work for a wider variety of languages.

本文旨在研究BERT等大型语言模型中的偏见问题，特别是种族偏见问题的度量和消除方法，使用了“Categorical Bias score”度量方法和两种消除方法，包括多语言模型和两个单语言模型的上下文词对齐方法，并对英语，德语，西班牙语，韩语，土耳其语和中文等多种语言进行验证和比较。结果表明，这些方法可以有效减轻种族偏见问题，但效果取决于该语言的NLP资源量。同时，本论文还验证了这些方法适用于更多种语言，如阿拉伯语和希腊语。

减轻BERT中的语言依赖性民族偏见