A critical problem in deep learning is that systems learn inappropriate
biases, resulting in their inability to perform well on minority groups. This
has led to the creation of multiple algorithms that endeavor to mitigate bias.
However, it is not clear how effective these methods are. This is because study
protocols differ among papers, systems are tested on datasets that fail to test
many forms of bias, and systems have access to hidden knowledge or are tuned
specifically to the test set. To address this, we introduce an improved
evaluation protocol, sensible metrics, and a new dataset, which enables us to
ask and answer critical questions about bias mitigation algorithms. We evaluate
seven state-of-the-art algorithms using the same network architecture and
hyperparameter selection policy across three benchmark datasets. We introduce a
new dataset called Biased MNIST that enables assessment of robustness to
multiple bias sources. We use Biased MNIST and a visual question answering
(VQA) benchmark to assess robustness to hidden biases. Rather than only tuning
to the test set distribution, we study robustness across different tuning
distributions, which is critical because for many applications the test
distribution may not be known during development. We find that algorithms
exploit hidden biases, are unable to scale to multiple forms of bias, and are
highly sensitive to the choice of tuning set. Based on our findings, we implore
the community to adopt more rigorous assessment of future bias mitigation
methods. All data, code, and results are publicly available at:
this https URL

本研究提出了一个改进的评估协议、合理的指标和一个新的数据集，以评估现有的 7 种最先进的减少偏差算法的有效性。研究发现，算法利用隐藏的偏见，无法适应多种偏向，而且对调整数据集的选择非常敏感。因此呼吁社区采用更严格的评估方法。