Massively multilingual sentence representations are trained on large corpora
of uncurated data, with a very imbalanced proportion of languages included in
the training. This may cause the models to grasp cultural values including
moral judgments from the high-resource languages and impose them on the
low-resource languages. The lack of data in certain languages can also lead to
developing random and thus potentially harmful beliefs. Both these issues can
negatively influence zero-shot cross-lingual model transfer and potentially
lead to harmful outcomes. Therefore, we aim to (1) detect and quantify these
issues by comparing different models in different languages, (2) develop
methods for improving undesirable properties of the models. Our initial
experiments using the multilingual model XLM-R show that indeed multilingual
LMs capture moral norms, even with potentially higher human-agreement than
monolingual ones. However, it is not yet clear to what extent these moral norms
differ between languages.

该论文旨在检测和量化高资源语言对低资源语言影响的问题，并寻求方法来提高跨语言模型转移的可靠性。作者使用跨语言模型 XLM-R 并发现，跨语言语言模型的确能够捕捉道德规范，但还不清楚这些规范在不同语言之间的差异程度。