Recently pre-trained multimodal models, such as CLIP, have shown exceptional
capabilities towards connecting images and natural language. The textual
representations in English can be desirably transferred to multilingualism and
support downstream multimodal tasks for different languages. Nevertheless, the
principle of multilingual fairness is rarely scrutinized: do multilingual
multimodal models treat languages equally? Are their performances biased
towards particular languages? To answer these questions, we view language as
the fairness recipient and introduce two new fairness notions, multilingual
individual fairness and multilingual group fairness, for pre-trained multimodal
models. Multilingual individual fairness requires that text snippets expressing
similar semantics in different languages connect similarly to images, while
multilingual group fairness requires equalized predictive performance across
languages. We characterize the extent to which pre-trained multilingual
vision-and-language representations are individually fair across languages.
However, extensive experiments demonstrate that multilingual representations do
not satisfy group fairness: (1) there is a severe multilingual accuracy
disparity issue; (2) the errors exhibit biases across languages conditioning
the group of people in the images, including race, gender and age.

本文通过引入多语言个体公平和多语言群体公平这两个公平性概念，分析了预训练的多模态模型是否公平地对待多种语言，结果表明预训练模型在各种语言上的表现差异很大，缺乏多语言群体公平特性，并且对于涉及种族、性别和年龄等方面的人物群体，模型存在一定的偏误。