Prediction models can improve efficiency by automating decisions such as the
approval of loan applications. However, they may inherit bias against protected
groups from the data they are trained on. This paper adds counterfactual
(simulated) ethnic bias to real data on mortgage application decisions, and
shows that this bias is replicated by a machine learning model (XGBoost) even
when ethnicity is not used as a predictive variable. Next, several other
de-biasing methods are compared: averaging over prohibited variables, taking
the most favorable prediction over prohibited variables (a novel method), and
jointly minimizing errors as well as the association between predictions and
prohibited variables. De-biasing can recover some of the original decisions,
but the results are sensitive to whether the bias is effected through a proxy.

通过对房屋贷款申请决策的真实数据添加反事实（模拟）种族偏见，本文展示了即使没有将族裔作为预测变量，机器学习模型（XGBoost）仍然会重复这种偏见。接下来比较了几种去偏方法：禁止变量的平均值、在禁止变量中选择最优预测（一种新方法）以及同时最小化误差和预测与禁止变量之间的关联。去偏可以恢复部分原始决策，但结果对于偏见是否通过代理方式产生是敏感的。

消除偏见的偏见决策模型比较：基于抵押贷款申请数据的方法对比

De-Biasing Models of Biased Decisions: A Comparison of Methods Using  Mortgage Application Data

Many extractive question answering models are trained to predict start and
end positions of answers. The choice of predicting answers as positions is
mainly due to its simplicity and effectiveness. In this study, we hypothesize
that when the distribution of the answer positions is highly skewed in the
training set (e.g., answers lie only in the k-th sentence of each passage), QA
models predicting answers as positions can learn spurious positional cues and
fail to give answers in different positions. We first illustrate this position
bias in popular extractive QA models such as BiDAF and BERT and thoroughly
examine how position bias propagates through each layer of BERT. To safely
deliver position information without position bias, we train models with
various de-biasing methods including entropy regularization and bias
ensembling. Among them, we found that using the prior distribution of answer
positions as a bias model is very effective at reducing position bias,
recovering the performance of BERT from 37.48% to 81.64% when trained on a
biased SQuAD dataset.

本研究发现在训练集中，答案位置分布呈高度偏斜时，使用位置预测作为答案的提取型问答模型容易学习到虚假的位置线索，并且在不同位置上无法提供正确的答案。因此，对于 BiDAF 和 BERT 等受到位置偏见影响的模型，我们实现了多种消除位置偏见的方法，发现使用答案先验分布作为偏见模型非常有效，在受到偏见的 SQuAD 数据集上，BERT 性能从 37.48% 提高到 81.64%。