Biased human decisions have consequential impacts across various domains, yielding unfair treatment of individuals and resulting in suboptimal outcomes for organizations and society. In recognition of this fact, organizations regularly design and deploy interventions aimed at mitigating these biases. However, measuring human decision biases remains an important but elusive task. Organizations are frequently concerned with mistaken decisions disproportionately affecting one group. In practice, however, this is typically not possible to assess due to the scarcity of a gold standard: a label that indicates what the correct decision would have been. In this work, we propose a machine learning-based framework to assess bias in human-generated decisions when gold standard labels are scarce. We provide theoretical guarantees and empirical evidence demonstrating the superiority of our method over existing alternatives. This proposed methodology establishes a foundation for transparency in human decision-making, carrying substantial implications for managerial duties, and offering potential for alleviating algorithmic biases when human decisions are used as labels to train algorithms.

本研究针对人类决策中的偏见问题，提出了一种机器学习框架来评估决策偏见，尤其是在缺乏黄金标准标签的情况下。通过理论证明和实证结果，展示了该方法在评估人类决策偏见方面的先进性，这为增强人类决策透明度提供了基础，并对管理决策和缓解算法偏见具有重要影响。

用偏见对抗偏见：评估人类偏见的机器学习方法