In machine learning fairness, training models which minimize disparity across
different sensitive groups often leads to diminished accuracy, a phenomenon
known as the fairness-accuracy trade-off. The severity of this trade-off
fundamentally depends on dataset characteristics such as dataset imbalances or
biases. Therefore using a uniform fairness requirement across datasets remains
questionable and can often lead to models with substantially low utility. To
address this, we present a computationally efficient approach to approximate
the fairness-accuracy trade-off curve tailored to individual datasets, backed
by rigorous statistical guarantees. By utilizing the You-Only-Train-Once (YOTO)
framework, our approach mitigates the computational burden of having to train
multiple models when approximating the trade-off curve. Moreover, we quantify
the uncertainty in our approximation by introducing confidence intervals around
this curve, offering a statistically grounded perspective on the acceptable
range of fairness violations for any given accuracy threshold. Our empirical
evaluation spanning tabular, image and language datasets underscores that our
approach provides practitioners with a principled framework for
dataset-specific fairness decisions across various data modalities.

机器学习公平性经常导致准确率下降，我们提出了一种基于数据集的高效方法来逼近公平性 - 准确性权衡曲线，并通过引入置信区间来量化逼近的不确定性，从而为各种数据模态下的数据集特定公平决策提供了一种有原则的框架。