Existing approaches to algorithmic fairness aim to ensure equitable outcomes if human decision-makers comply perfectly with algorithmic decisions. However, perfect compliance with the algorithm is rarely a reality or even a desirable outcome in human-AI collaboration. Yet, recent studies have shown that selective compliance with fair algorithms can amplify discrimination relative to the prior human policy. As a consequence, ensuring equitable outcomes requires fundamentally different algorithmic design principles that ensure robustness to the decision-maker's (a priori unknown) compliance pattern. We define the notion of compliance-robustly fair algorithmic recommendations that are guaranteed to (weakly) improve fairness in decisions, regardless of the human's compliance pattern. We propose a simple optimization strategy to identify the best performance-improving compliance-robustly fair policy. However, we show that it may be infeasible to design algorithmic recommendations that are simultaneously fair in isolation, compliance-robustly fair, and more accurate than the human policy; thus, if our goal is to improve the equity and accuracy of human-AI collaboration, it may not be desirable to enforce traditional fairness constraints.

现有的算法公平性方法旨在确保人类决策者与算法决策完全一致，然而，在人工智能与人类协作中，与算法的完全一致很少实现或者甚至是不可取的。我们提出了“合规性鲁棒公平算法推荐”概念，它能够保证在决策中改善公平性，无论人类的合规模式如何。然而，我们发现设计同时具备独立公平性、合规性鲁棒公平性和比人类决策更准确的算法推荐可能是不可行的，因此，如果我们的目标是改善人工智能与人类协作的公平性和准确性，可能不应强制执行传统的公平性约束。

重新思考人工智能与人类协作的公平性