Internet search affects people's cognition of the world, so mitigating biases
in search results and learning fair models is imperative for social good. We
study a unique gender bias in image search in this work: the search images are
often gender-imbalanced for gender-neutral natural language queries. We
diagnose two typical image search models, the specialized model trained on
in-domain datasets and the generalized representation model pre-trained on
massive image and text data across the internet. Both models suffer from severe
gender bias. Therefore, we introduce two novel debiasing approaches: an
in-processing fair sampling method to address the gender imbalance issue for
training models, and a post-processing feature clipping method base on mutual
information to debias multimodal representations of pre-trained models.
Extensive experiments on MS-COCO and Flickr30K benchmarks show that our methods
significantly reduce the gender bias in image search models.

本文研究互联网搜索中出现的性别偏见，尤其是在图像搜索中的性别失衡问题。为此，我们提出了两种消除偏见的方法：采用处理中的合理采样方法解决训练模型时的性别失衡问题，以及采用基于互信息的后处理特征裁剪方法来消除预训练模型的多模式表示中的偏见。通过在 MS-COCO 和 Flickr30K 基准测试上进行的大量实验，证明了我们的方法显著降低了图像搜索模型中的性别偏见。