The error of an estimator can be decomposed into a (statistical) bias term, a variance term, and an irreducible noise term. When we do bias analysis, formally we are asking the question: "how good are the predictions?" The role of bias in the error decomposition is clear: if we trust the labels/targets, then we would want the estimator to have as low bias as possible, in order to minimize error. Fair machine learning is concerned with the question: "Are the predictions equally good for different demographic/social groups?" This has naturally led to a variety of fairness metrics that compare some measure of statistical bias on subsets corresponding to socially privileged and socially disadvantaged groups. In this paper we propose a new family of performance measures based on group-wise parity in variance. We demonstrate when group-wise statistical bias analysis gives an incomplete picture, and what group-wise variance analysis can tell us in settings that differ in the magnitude of statistical bias. We develop and release an open-source library that reconciles uncertainty quantification techniques with fairness analysis, and use it to conduct an extensive empirical analysis of our variance-based fairness measures on standard benchmarks.

本文讨论了估计器误差分解中的偏差项、方差项和不可避免噪声项， 研究了社会特权和社会劣势群体的公平性问题，并提出了一种基于组间方差的性能度量方法。同时，研发了一个开源库，将不确定性量化技术与公平性分析融合，并在标准基准测试上对基于方差的公平性度量方法进行了全面的实证分析。

公平与稳定性：估计方差是朋友还是敌人？