We address the challenge of generating fair and unbiased image retrieval
results given neutral textual queries (with no explicit gender or race
connotations), while maintaining the utility (performance) of the underlying
vision-language (VL) model. Previous methods aim to disentangle learned
representations of images and text queries from gender and racial
characteristics. However, we show these are inadequate at alleviating bias for
the desired equal representation result, as there usually exists test-time bias
in the target retrieval set. So motivated, we introduce a straightforward
technique, Post-hoc Bias Mitigation (PBM), that post-processes the outputs from
the pre-trained vision-language model. We evaluate our algorithm on real-world
image search datasets, Occupation 1 and 2, as well as two large-scale
image-text datasets, MS-COCO and Flickr30k. Our approach achieves the lowest
bias, compared with various existing bias-mitigation methods, in text-based
image retrieval result while maintaining satisfactory retrieval performance.
The source code is publicly available at
https://anonymous.4open.science/r/Fair_Text_based_Image_Retrieval-D8B2.

本篇论文针对文本查询时图像检索结果存在的性别偏见和种族偏见，提出了一种简单直接的后处理偏见缓解技术，可以在保持图像检索性能不变的情况下，实现对性别和种族的平等代表性，最终得到比其他现有同类方法更低的偏见度和令人满意的图像检索表现。

降低测试时间偏差以实现公平图像检索

Mitigating Test-Time Bias for Fair Image Retrieval

There is growing interest in searching for information from large video
corpora. Prior works have studied relevant tasks, such as text-based video
retrieval, moment retrieval, video summarization, and video captioning in
isolation, without an end-to-end setup that can jointly search from video
corpora and generate summaries. Such an end-to-end setup would allow for many
interesting applications, e.g., a text-based search that finds a relevant video
from a video corpus, extracts the most relevant moment from that video, and
segments the moment into important steps with captions. To address this, we
present the HiREST (HIerarchical REtrieval and STep-captioning) dataset and
propose a new benchmark that covers hierarchical information retrieval and
visual/textual stepwise summarization from an instructional video corpus.
HiREST consists of 3.4K text-video pairs from an instructional video dataset,
where 1.1K videos have annotations of moment spans relevant to text query and
breakdown of each moment into key instruction steps with caption and timestamps
(totaling 8.6K step captions). Our hierarchical benchmark consists of video
retrieval, moment retrieval, and two novel moment segmentation and step
captioning tasks. In moment segmentation, models break down a video moment into
instruction steps and identify start-end boundaries. In step captioning, models
generate a textual summary for each step. We also present starting point
task-specific and end-to-end joint baseline models for our new benchmark. While
the baseline models show some promising results, there still exists large room
for future improvement by the community. Project website:
this https URL

本研究介绍了 HiREST 数据集和基准测试，涵盖了从以下视频语料库中检索单元（瞬间）并分层检索高亮步骤的任务。HiREST 包括来自教学视频数据集的 3.4K 个文本视频对，其中 1.1K 个视频具有与文本查询相关的瞬间跨度注释，并将每个瞬间的重点说明步骤与时间戳（总计 8.6K 个步骤说明）的分解。本研究还为我们的新基准测试提供了一个起点任务特定的和端到端的联合基线模型。虽然基线模型显示出了一些有希望的结果，但仍然存在大量改进空间。