Recent work in fairness in machine learning has proposed adjusting for
fairness by equalizing accuracy metrics across groups and has also studied how
datasets affected by historical prejudices may lead to unfair decision
policies. We connect these lines of work and study the residual unfairness that
arises when a fairness-adjusted predictor is not actually fair on the target
population due to systematic censoring of training data by existing biased
policies. This scenario is particularly common in the same applications where
fairness is a concern. We characterize theoretically the impact of such
censoring on standard fairness metrics for binary classifiers and provide
criteria for when residual unfairness may or may not appear. We prove that,
under certain conditions, fairness-adjusted classifiers will in fact induce
residual unfairness that perpetuates the same injustices, against the same
groups, that biased the data to begin with, thus showing that even
state-of-the-art fair machine learning can have a "bias in, bias out" property.
When certain benchmark data is available, we show how sample reweighting can
estimate and adjust fairness metrics while accounting for censoring. We use
this to study the case of Stop, Question, and Frisk (SQF) and demonstrate that
attempting to adjust for fairness perpetuates the same injustices that the
policy is infamous for.

研究公平机器学习时，为达到公平，可以通过在不同群体之间平衡度量指标，并研究历史偏见对数据集的影响，然而在训练有偏倚策略的数据时，可能会导致调整后的预测器在目标人群中并不公平，从而引起剩余不公平。 使用样本重新加权来估算和调整公平度量标准，以制定出更公平的机器学习算法。