Multi-modal search engines have experienced significant growth and widespread
use in recent years, making them the second most common internet use. While
search engine systems offer a range of services, the image search field has
recently become a focal point in the information retrieval community, as the
adage goes, "a picture is worth a thousand words". Although popular search
engines like Google excel at image search accuracy and agility, there is an
ongoing debate over whether their search results can be biased in terms of
gender, language, demographics, socio-cultural aspects, and stereotypes. This
potential for bias can have a significant impact on individuals' perceptions
and influence their perspectives.
In this paper, we present our study on bias and fairness in web search, with
a focus on keyword-based image search. We first discuss several kinds of biases
that exist in search systems and why it is important to mitigate them. We
narrow down our study to assessing and mitigating occupational stereotypes in
image search, which is a prevalent fairness issue in image retrieval. For the
assessment of stereotypes, we take gender as an indicator. We explore various
open-source and proprietary APIs for gender identification from images. With
these, we examine the extent of gender bias in top-tanked image search results
obtained for several occupational keywords. To mitigate the bias, we then
propose a fairness-aware re-ranking algorithm that optimizes (a) relevance of
the search result with the keyword and (b) fairness w.r.t genders identified.
We experiment on 100 top-ranked images obtained for 10 occupational keywords
and consider random re-ranking and re-ranking based on relevance as baselines.
Our experimental results show that the fairness-aware re-ranking algorithm
produces rankings with better fairness scores and competitive relevance scores
than the baselines.

本研究聚焦于关键词图像搜索领域中的偏见和公平性问题，评估并减轻搜索结果中的性别职业刻板印象。通过使用开源和专有的 API 来确定图像的性别，我们提出了一种考虑到与指定关键词的相关性和公平性的公平感知排序算法，并在实验中证明其显著地提高了公平性分数和相关性分数。