Vision Language Models (VLMs) such as CLIP are powerful models; however they
can exhibit unwanted biases, making them less safe when deployed directly in
applications such as text-to-image, text-to-video retrievals, reverse search,
or classification tasks. In this work, we propose a novel framework to generate
synthetic counterfactual images to create a diverse and balanced dataset that
can be used to fine-tune CLIP. Given a set of diverse synthetic base images
from text-to-image models, we leverage off-the-shelf segmentation and
inpainting models to place humans with diverse visual appearances in context.
We show that CLIP trained on such datasets learns to disentangle the human
appearance from the context of an image, i.e., what makes a doctor is not
correlated to the person's visual appearance, like skin color or body type, but
to the context, such as background, the attire they are wearing, or the objects
they are holding. We demonstrate that our fine-tuned CLIP model, $CF_\alpha$,
improves key fairness metrics such as MaxSkew, MinSkew, and NDKL by 40-66\% for
image retrieval tasks, while still achieving similar levels of performance in
downstream tasks. We show that, by design, our model retains maximal
compatibility with the original CLIP models, and can be easily controlled to
support different accuracy versus fairness trade-offs in a plug-n-play fashion.

通过生成合成的反事实图像集合，采用脱节模型训练方法来改善 Vision Language Models（VLMs）的公平性和性能。

他们都是医生：合成各种对抗偏见的反事实案例

They're All Doctors: Synthesizing Diverse Counterfactuals to Mitigate  Associative Bias

Computer-based decision systems are widely used to automate decisions in many
aspects of everyday life, which include sensitive areas like hiring, loaning
and even criminal sentencing. A decision pipeline heavily relies on large
volumes of historical real-world data for training its models. However,
historical training data often contains gender, racial or other biases which
are propagated to the trained models influencing computer-based decisions. In
this work, we propose a robust methodology that guarantees the removal of
unwanted biases while maximally preserving classification utility. Our approach
can always achieve this in a model-independent way by deriving from real-world
data the asymptotic dataset that uniquely encodes demographic parity and
realism. As a proof-of-principle, we deduce from public census records such an
asymptotic dataset from which synthetic samples can be generated to train
well-established classifiers. Benchmarking the generalization capability of
these classifiers trained on our synthetic data, we confirm the absence of any
explicit or implicit bias in the computer-aided decision.

计算机决策系统在许多日常生活的方面得到广泛应用，但其中可能存在性别、种族或其他偏见问题。本研究提出一种鲁棒的方法，通过真实世界数据提取出表征人口平衡和真实性的数据集，用于训练分类器，并测试其泛化能力，证实计算机辅助决策中不存在明示或隐含偏见。

人口平等：缓解现实世界数据中的偏见

Demographic Parity: Mitigating Biases in Real-World Data

Benchmark datasets play an important role in evaluating Natural Language
Understanding (NLU) models. However, shortcuts -- unwanted biases in the
benchmark datasets -- can damage the effectiveness of benchmark datasets in
revealing models' real capabilities. Since shortcuts vary in coverage,
productivity, and semantic meaning, it is challenging for NLU experts to
systematically understand and avoid them when creating benchmark datasets. In
this paper, we develop a visual analytics system, ShortcutLens, to help NLU
experts explore shortcuts in NLU benchmark datasets. The system allows users to
conduct multi-level exploration of shortcuts. Specifically, Statistics View
helps users grasp the statistics such as coverage and productivity of shortcuts
in the benchmark dataset. Template View employs hierarchical and interpretable
templates to summarize different types of shortcuts. Instance View allows users
to check the corresponding instances covered by the shortcuts. We conduct case
studies and expert interviews to evaluate the effectiveness and usability of
the system. The results demonstrate that ShortcutLens supports users in gaining
a better understanding of benchmark dataset issues through shortcuts, inspiring
them to create challenging and pertinent benchmark datasets.

本文提出了一种名为 ShortcutLens 的可视化分析系统，帮助 NLU 专家探索 NLU 基准数据集中的捷径，该系统允许用户进行多层次的捷径探索，包括统计视图、模板视图和实例视图，研究结果表明 ShortcutLens 支持用户更好地了解基准数据集中的捷径问题，激励他们创建具有挑战性和相关性的基准数据集。

ShortcutLens：一种用于探索自然语言理解数据集中快捷方式的可视化分析方法

ShortcutLens: A Visual Analytics Approach for Exploring Shortcuts in Natural Language Understanding Dataset

Today, AI is increasingly being used in many high-stakes decision-making
applications in which fairness is an important concern. Already, there are many
examples of AI being biased and making questionable and unfair decisions. The
AI research community has proposed many methods to measure and mitigate
unwanted biases, but few of them involve inputs from human policy makers. We
argue that because different fairness criteria sometimes cannot be
simultaneously satisfied, and because achieving fairness often requires
sacrificing other objectives such as model accuracy, it is key to acquire and
adhere to human policy makers' preferences on how to make the tradeoff among
these objectives. In this paper, we propose a framework and some exemplar
methods for eliciting such preferences and for optimizing an AI model according
to these preferences.

本文提出了一种用于优化 AI 模型的框架和一些示例方法，根据人类政策制定者的偏好来平衡公平性、模型准确性等目标之间的平衡，以此来减少偏差和不公平。