Are language models culturally biased? It is important that language models conform to the cultural aspects of the communities they serve. However, we show in this paper that language models suffer from a significant bias towards Western culture when handling and generating text in Arabic, often preferring, and producing Western-fitting content as opposed to the relevant Arab content. We quantify this bias through a likelihood scoring-based metric using naturally occurring contexts that we collect from online social media. Our experiments reveal that both Arabic monolingual and multilingual models exhibit bias towards Western culture in eight different cultural aspects: person names, food, clothing, location, literature, beverage, religion, and sports. Models also tend to exhibit more bias when prompted with Arabic sentences that are more linguistically aligned with English. These findings raise concerns about the cultural relevance of current language models. Our analyses show that providing culture-indicating tokens or culturally-relevant demonstrations to the model can help in debiasing.

本文探讨语言模型是否存在文化偏见，研究发现目前的语言模型在处理和生成阿拉伯语文本时存在明显的西方文化偏见，特别是在人名、食物、服装、地点、文学、饮料、宗教和体育等八个方面。同时，研究表明，向模型提供文化指示符或相关文化示范可以帮助消除偏见。

测量大型语言模型中的文化偏见：祈祷后喝啤酒？