Deep models trained on large amounts of data often incorporate implicit
biases present during training time. If later such a bias is discovered during
inference or deployment, it is often necessary to acquire new data and retrain
the model. This behavior is especially problematic in critical areas such as
autonomous driving or medical decision-making. In these scenarios, new data is
often expensive and hard to come by. In this work, we present a method based on
change penalization that takes a pre-trained model and adapts the weights to
mitigate a previously detected bias. We achieve this by tuning a
zero-initialized copy of a frozen pre-trained network. Our method needs very
few, in extreme cases only a single, examples that contradict the bias to
increase performance. Additionally, we propose an early stopping criterion to
modify baselines and reduce overfitting. We evaluate our approach on a
well-known bias in skin lesion classification and three other datasets from the
domain shift literature. We find that our approach works especially well with
very few images. Simple fine-tuning combined with our early stopping also leads
to performance benefits for a larger number of tuning samples.

基于改变惩罚的方法来减轻模型中的隐性偏见，需要很少的对立示例并结合提前停止准则以增加性能。

通过调整并对变化进行惩罚来减少预训练模型中的偏差

Reducing Bias in Pre-trained Models by Tuning while Penalizing Change

We present Face0, a novel way to instantaneously condition a text-to-image
generation model on a face, in sample time, without any optimization procedures
such as fine-tuning or inversions. We augment a dataset of annotated images
with embeddings of the included faces and train an image generation model, on
the augmented dataset. Once trained, our system is practically identical at
inference time to the underlying base model, and is therefore able to generate
images, given a user-supplied face image and a prompt, in just a couple of
seconds. Our method achieves pleasing results, is remarkably simple, extremely
fast, and equips the underlying model with new capabilities, like controlling
the generated images both via text or via direct manipulation of the input face
embeddings. In addition, when using a fixed random vector instead of a face
embedding from a user supplied image, our method essentially solves the problem
of consistent character generation across images. Finally, while requiring
further research, we hope that our method, which decouples the model's textual
biases from its biases on faces, might be a step towards some mitigation of
biases in future text-to-image models.

本论文利用 Face0 增强了图像生成模型的性能并且解决了一些基于文本生成图像时的偏差问题，可以通过文本或直接操作输入人脸嵌入来控制生成的图像。

在人脸上即时调整文本到图像模型

Face0: Instantaneously Conditioning a Text-to-Image Model on a Face

Language representations are known to carry stereotypical biases and, as a
result, lead to biased predictions in downstream tasks. While existing methods
are effective at mitigating biases by linear projection, such methods are too
aggressive: they not only remove bias, but also erase valuable information from
word embeddings. We develop new measures for evaluating specific information
retention that demonstrate the tradeoff between bias removal and information
retention. To address this challenge, we propose OSCaR (Orthogonal Subspace
Correction and Rectification), a bias-mitigating method that focuses on
disentangling biased associations between concepts instead of removing concepts
wholesale. Our experiments on gender biases show that OSCaR is a well-balanced
approach that ensures that semantic information is retained in the embeddings
and bias is also effectively mitigated.

本文提出了一种名为 OSCaR 的新的降低偏见的方法，该方法专注于解开概念之间的偏见关联而非整体去除概念。实验结果表明，OSCaR 方法保证了嵌入中的语义信息被保留且能够有效地缓解偏见，特别是在性别偏见的情况下表现出良好的平衡性。