Data practices shape research and practice on fairness in machine learning
(fair ML). Critical data studies offer important reflections and critiques for
the responsible advancement of the field by highlighting shortcomings and
proposing recommendations for improvement. In this work, we present a
comprehensive analysis of fair ML datasets, demonstrating how unreflective yet
common practices hinder the reach and reliability of algorithmic fairness
findings. We systematically study protected information encoded in tabular
datasets and their usage in 280 experiments across 142 publications.
Our analyses identify three main areas of concern: (1) a \textbf{lack of
representation for certain protected attributes} in both data and evaluations;
(2) the widespread \textbf{exclusion of minorities} during data preprocessing;
and (3) \textbf{opaque data processing} threatening the generalization of
fairness research. By conducting exemplary analyses on the utilization of
prominent datasets, we demonstrate how unreflective data decisions
disproportionately affect minority groups, fairness metrics, and resultant
model comparisons. Additionally, we identify supplementary factors such as
limitations in publicly available data, privacy considerations, and a general
lack of awareness, which exacerbate these challenges. To address these issues,
we propose a set of recommendations for data usage in fairness research
centered on transparency and responsible inclusion. This study underscores the
need for a critical reevaluation of data practices in fair ML and offers
directions to improve both the sourcing and usage of datasets.

数据实践塑造了公平机器学习研究和实践。关键数据研究通过指出不足并提出改进建议，为该领域的负责前进提供了重要的反思和批评。本研究对公平机器学习数据集进行了全面分析，展示了不经思考的常见实践如何阻碍算法公平研究的覆盖率和可靠性。我们对表格数据集中编码的保护信息以及在 142 篇出版物的 280 个实验中的使用进行了系统研究，发现了三个主要问题：（1）数据和评估中某些保护属性的缺乏代表性；（2）数据预处理过程中普遍排除少数群体；以及（3）威胁公平性研究泛化的模糊数据处理。通过在显著数据集的利用上进行示范性分析，我们展示了不经思考的数据决策如何不成比例地影响少数群体、公平度量和模型比较结果。此外，我们还发现了公开可用数据的限制、隐私考虑以及普遍缺乏意识等补充因素，加剧了这些挑战。为了解决这些问题，我们提出了一套以透明和负责任包容为核心的数据使用建议。本研究强调了对公平机器学习中数据实践进行关键重新评估的必要性，并提供了改善数据的获取和使用的方向。

懒散的数据处理方式对公正研究的危害

Lazy Data Practices Harm Fairness Research

In recent years, machine learning algorithms have become ubiquitous in a
multitude of high-stakes decision-making applications. The unparalleled ability
of machine learning algorithms to learn patterns from data also enables them to
incorporate biases embedded within. A biased model can then make decisions that
disproportionately harm certain groups in society -- limiting their access to
financial services, for example. The awareness of this problem has given rise
to the field of Fair ML, which focuses on studying, measuring, and mitigating
unfairness in algorithmic prediction, with respect to a set of protected groups
(e.g., race or gender). However, the underlying causes for algorithmic
unfairness still remain elusive, with researchers divided between blaming
either the ML algorithms or the data they are trained on. In this work, we
maintain that algorithmic unfairness stems from interactions between models and
biases in the data, rather than from isolated contributions of either of them.
To this end, we propose a taxonomy to characterize data bias and we study a set
of hypotheses regarding the fairness-accuracy trade-offs that fairness-blind ML
algorithms exhibit under different data bias settings. On our real-world
account-opening fraud use case, we find that each setting entails specific
trade-offs, affecting fairness in expected value and variance -- the latter
often going unnoticed. Moreover, we show how algorithms compare differently in
terms of accuracy and fairness, depending on the biases affecting the data.
Finally, we note that under specific data bias conditions, simple
pre-processing interventions can successfully balance group-wise error rates,
while the same techniques fail in more complex settings.

机器学习算法应用广泛，但也可能存在偏见和不公平。本研究提出了一种刻画数据偏见的分类法，并探究了算法的不公平和精度之间的权衡。实证研究表明，在不同的数据偏见设置下，算法的公平性和精度会受到不同的影响，而简单的预处理干预技术可以提高算法的公平性。