Machine learning model bias can arise from dataset composition: sensitive
features correlated to the learning target disturb the model decision rule and
lead to performance differences along the features. Existing de-biasing work
captures prominent and delicate image features which are traceable in model
latent space, like colors of digits or background of animals. However, using
the latent space is not sufficient to understand all dataset feature
correlations. In this work, we propose a framework to extract feature clusters
in a dataset based on image descriptions, allowing us to capture both subtle
and coarse features of the images. The feature co-occurrence pattern is
formulated and correlation is measured, utilizing a human-in-the-loop for
examination. The analyzed features and correlations are human-interpretable, so
we name the method Common-Sense Bias Discovery (CSBD). Having exposed sensitive
correlations in a dataset, we demonstrate that downstream model bias can be
mitigated by adjusting image sampling weights, without requiring a sensitive
group label supervision. Experiments show that our method discovers novel
biases on multiple classification tasks for two benchmark image datasets, and
the intervention outperforms state-of-the-art unsupervised bias mitigation
methods.

基于图像描述提取特征聚类的框架，发现数据集中的敏感关联性，通过调整图像采样权重减轻下游模型偏差问题。

分类任务的常识偏见发现与缓解

Common-Sense Bias Discovery and Mitigation for Classification Tasks

A critical challenge for tumour segmentation models is the ability to adapt
to diverse clinical settings, particularly when applied to poor-quality
neuroimaging data. The uncertainty surrounding this adaptation stems from the
lack of representative datasets, leaving top-performing models without exposure
to common artifacts found in MRI data throughout Sub-Saharan Africa (SSA). We
replicated a framework that secured the 2nd position in the 2022 BraTS
competition to investigate the impact of dataset composition on model
performance and pursued four distinct approaches through training a model with:
1) BraTS-Africa data only (train_SSA, N=60), 2) BraTS-Adult Glioma data only
(train_GLI, N=1251), 3) both datasets together (train_ALL, N=1311), and 4)
through further training the train_GLI model with BraTS-Africa data
(train_ftSSA). Notably, training on a smaller low-quality dataset alone
(train_SSA) yielded subpar results, and training on a larger high-quality
dataset alone (train_GLI) struggled to delineate oedematous tissue in the
low-quality validation set. The most promising approach (train_ftSSA) involved
pre-training a model on high-quality neuroimages and then fine-tuning it on the
smaller, low-quality dataset. This approach outperformed the others, ranking
second in the MICCAI BraTS Africa global challenge external testing phase.
These findings underscore the significance of larger sample sizes and broad
exposure to data in improving segmentation performance. Furthermore, we
demonstrated that there is potential for improving such models by fine-tuning
them with a wider range of data locally.

通过对不同数据集进行训练，本研究发现，在低质量数据环境下，将模型预训练于高质量神经影像数据，并在较小规模低质量数据上进行微调，可以显著提高肿瘤分割模型的性能。

缩小差距：将最先进的 U-Net 模型推广至撒哈拉以南非洲人群

Bridging the Gap: Generalising State-of-the-Art U-Net Models to  Sub-Saharan African Populations

Background: With the ever-increasing amount of medical imaging data, the
demand for algorithms to assist clinicians has amplified. Unsupervised anomaly
detection (UAD) models promise to aid in the crucial first step of disease
detection. While previous studies have thoroughly explored fairness in
supervised models in healthcare, for UAD, this has so far been unexplored.
Methods: In this study, we evaluated how dataset composition regarding
subgroups manifests in disparate performance of UAD models along multiple
protected variables on three large-scale publicly available chest X-ray
datasets. Our experiments were validated using two state-of-the-art UAD models
for medical images. Finally, we introduced a novel subgroup-AUROC (sAUROC)
metric, which aids in quantifying fairness in machine learning.
Findings: Our experiments revealed empirical "fairness laws" (similar to
"scaling laws" for Transformers) for training-dataset composition: Linear
relationships between anomaly detection performance within a subpopulation and
its representation in the training data. Our study further revealed performance
disparities, even in the case of balanced training data, and compound effects
that exacerbate the drop in performance for subjects associated with multiple
adversely affected groups.
Interpretation: Our study quantified the disparate performance of UAD models
against certain demographic subgroups. Importantly, we showed that this
unfairness cannot be mitigated by balanced representation alone. Instead, the
representation of some subgroups seems harder to learn by UAD models than that
of others. The empirical fairness laws discovered in our study make disparate
performance in UAD models easier to estimate and aid in determining the most
desirable dataset composition.

本研究评估了数据集构成对多个保护变量上的无监督异常检测模型性能造成的影响，并引入了新的子组 - AUROC 度量方法，结果显示数据集构成对 UAD 模型的不公平性性能有显著影响。

（可预测的）无监督异常检测中的性能偏差

(Predictable) Performance Bias in Unsupervised Anomaly Detection

Computers are increasingly used to make decisions that have significant
impact in people's lives. Often, these predictions can affect different
population subgroups disproportionately. As a result, the issue of fairness has
received much recent interest, and a number of fairness-enhanced classifiers
and predictors have appeared in the literature. This paper seeks to study the
following questions: how do these different techniques fundamentally compare to
one another, and what accounts for the differences? Specifically, we seek to
bring attention to many under-appreciated aspects of such fairness-enhancing
interventions. Concretely, we present the results of an open benchmark we have
developed that lets us compare a number of different algorithms under a variety
of fairness measures, and a large number of existing datasets. We find that
although different algorithms tend to prefer specific formulations of fairness
preservations, many of these measures strongly correlate with one another. In
addition, we find that fairness-preserving algorithms tend to be sensitive to
fluctuations in dataset composition (simulated in our benchmark by varying
training-test splits), indicating that fairness interventions might be more
brittle than previously thought.

本篇研究通过开发公开基准来对不同的公平性增强算法进行比较，并发现许多公平性措施之间具有强烈的相关性，但这些措施也对数据集构成的波动敏感，这表明公平干预可能比以前认为的更脆弱。