Measures of algorithmic fairness are usually discussed in the context of binary decisions. We extend the approach to continuous scores. So far, ROC-based measures have mainly been suggested for this purpose. Other existing methods depend heavily on the distribution of scores, are unsuitable for ranking tasks, or their effect sizes are not interpretable. Here, we propose a distributionally invariant version of fairness measures for continuous scores with a reasonable interpretation based on the Wasserstein distance. Our measures are easily computable and well suited for quantifying and interpreting the strength of group disparities as well as for comparing biases across different models, datasets, or time points. We derive a link between the different families of existing fairness measures for scores and show that the proposed distributionally invariant fairness measures outperform ROC-based fairness measures because they are more explicit and can quantify significant biases that ROC-based fairness measures miss. Finally, we demonstrate their effectiveness through experiments on the most commonly used fairness benchmark datasets.

通过分布不变公平性度量，我们提出了一套对连续计分进行公平性度量的方法，该方法基于Wasserstein距离，具有合理的解释性，并且适用于量化和解释群组差异的强度，以及在不同模型、数据集或时间点之间比较偏差。我们还证明了这些度量的有效性，通过对常用公平性基准数据集的实验验证。

连续分数的可解释分布不变公平度量