Foundation models like CLIP allow zero-shot transfer on various tasks without
additional training data. Yet, the zero-shot performance is less competitive
than a fully supervised one. Thus, to enhance the performance, fine-tuning and
ensembling are also commonly adopted to better fit the downstream tasks.
However, we argue that such prior work has overlooked the inherent biases in
foundation models. Due to the highly imbalanced Web-scale training set, these
foundation models are inevitably skewed toward frequent semantics, and thus the
subsequent fine-tuning or ensembling is still biased. In this study, we
systematically examine the biases in foundation models and demonstrate the
efficacy of our proposed Generalized Logit Adjustment (GLA) method. Note that
bias estimation in foundation models is challenging, as most pre-train data
cannot be explicitly accessed like in traditional long-tailed classification
tasks. To this end, GLA has an optimization-based bias estimation approach for
debiasing foundation models. As our work resolves a fundamental flaw in the
pre-training, the proposed GLA demonstrates significant improvements across a
diverse range of tasks: it achieves 1.5 pp accuracy gains on ImageNet, an large
average improvement (1.4-4.6 pp) on 11 few-shot datasets, 2.4 pp gains on
long-tailed classification. Codes are in https://github.com/BeierZhu/GLA.

该研究系统地检查了基础模型中的偏见，并展示了我们提出的广义对数调整（GLA）方法的有效性。GLA 在解决预训练中的基本缺陷方面取得了重大改进，对于各种任务都取得了显著的提高。

广义逻辑调整：在基础模型中消除标签偏差以校准微调模型

Generalized Logit Adjustment: Calibrating Fine-tuned Models by Removing  Label Bias in Foundation Models

Training and evaluation of fair classifiers is a challenging problem. This is
partly due to the fact that most fairness metrics of interest depend on both
the sensitive attribute information and label information of the data points.
In many scenarios it is not possible to collect large datasets with such
information. An alternate approach that is commonly used is to separately train
an attribute classifier on data with sensitive attribute information, and then
use it later in the ML pipeline to evaluate the bias of a given classifier.
While such decoupling helps alleviate the problem of demographic scarcity, it
raises several natural questions such as: how should the attribute classifier
be trained?, and how should one use a given attribute classifier for accurate
bias estimation? In this work we study this question from both theoretical and
empirical perspectives.
We first experimentally demonstrate that the test accuracy of the attribute
classifier is not always correlated with its effectiveness in bias estimation
for a downstream model. In order to further investigate this phenomenon, we
analyze an idealized theoretical model and characterize the structure of the
optimal classifier. Our analysis has surprising and counter-intuitive
implications where in certain regimes one might want to distribute the error of
the attribute classifier as unevenly as possible among the different subgroups.
Based on our analysis we develop heuristics for both training and using
attribute classifiers for bias estimation in the data scarce regime. We
empirically demonstrate the effectiveness of our approach on real and simulated
data.

研究公平分类器培训和评估的挑战，探讨对数据点的敏感属性信息和标签信息的公平性指标依赖性，以及在数据缺乏的情况下如何训练和使用属性分类器进行偏差估计。

评估不确定和不完整信息下机器学习模型的公平性

Evaluating Fairness of Machine Learning Models Under Uncertain and  Incomplete Information

We investigate meta-learning procedures in the setting of stochastic linear
bandits tasks. The goal is to select a learning algorithm which works well on
average over a class of bandits tasks, that are sampled from a
task-distribution. Inspired by recent work on learning-to-learn linear
regression, we consider a class of bandit algorithms that implement a
regularized version of the well-known OFUL algorithm, where the regularization
is a square euclidean distance to a bias vector. We first study the benefit of
the biased OFUL algorithm in terms of regret minimization. We then propose two
strategies to estimate the bias within the learning-to-learn setting. We show
both theoretically and experimentally, that when the number of tasks grows and
the variance of the task-distribution is small, our strategies have a
significant advantage over learning the tasks in isolation.

研究在随机线性赌博任务中的元学习过程，通过从任务分布中采样一类赌博任务来选择平均表现良好的学习算法，该文章考虑了实现 OFUL 算法的一类赌博算法，其中正则化是一个到偏置向量的平方欧几里得距离。我们首先研究了 OFUL 算法偏置的优点，并提出两种估计学习过程中偏差的策略。当任务数增加且任务分布的方差很小时，理论和实验都表明，我们的策略在学习隔离任务方面具有显着优势。

随机线性赌器元学习

Meta-learning with Stochastic Linear Bandits

Most NLP datasets are not annotated with protected attributes such as gender,
making it difficult to measure classification bias using standard measures of
fairness (e.g., equal opportunity). However, manually annotating a large
dataset with a protected attribute is slow and expensive. Instead of annotating
all the examples, can we annotate a subset of them and use that sample to
estimate the bias? While it is possible to do so, the smaller this annotated
sample is, the less certain we are that the estimate is close to the true bias.
In this work, we propose using Bernstein bounds to represent this uncertainty
about the bias estimate as a confidence interval. We provide empirical evidence
that a 95% confidence interval derived this way consistently bounds the true
bias. In quantifying this uncertainty, our method, which we call
Bernstein-bounded unfairness, helps prevent classifiers from being deemed
biased or unbiased when there is insufficient evidence to make either claim.
Our findings suggest that the datasets currently used to measure specific
biases are too small to conclusively identify bias except in the most egregious
cases. For example, consider a co-reference resolution system that is 5% more
accurate on gender-stereotypical sentences -- to claim it is biased with 95%
confidence, we need a bias-specific dataset that is 3.8 times larger than
WinoBias, the largest available.

使用 Bernstein 界限来表示关于偏差估计的不确定性，从而帮助防止分类器被认为是有偏见或无偏见，当没有足够的证据来做出任何一个主张。