Learning to defer (L2D) aims to improve human-AI collaboration systems by
learning how to defer decisions to humans when they are more likely to be
correct than an ML classifier. Existing research in L2D overlooks key aspects
of real-world systems that impede its practical adoption, namely: i) neglecting
cost-sensitive scenarios, where type 1 and type 2 errors have different costs;
ii) requiring concurrent human predictions for every instance of the training
dataset and iii) not dealing with human work capacity constraints. To address
these issues, we propose the deferral under cost and capacity constraints
framework (DeCCaF). DeCCaF is a novel L2D approach, employing supervised
learning to model the probability of human error under less restrictive data
requirements (only one expert prediction per instance) and using constraint
programming to globally minimize the error cost subject to workload
limitations. We test DeCCaF in a series of cost-sensitive fraud detection
scenarios with different teams of 9 synthetic fraud analysts, with individual
work capacity constraints. The results demonstrate that our approach performs
significantly better than the baselines in a wide array of scenarios, achieving
an average 8.4% reduction in the misclassification cost.

学员分享学习（L2D）旨在改善人工智能协作系统，通过学习如何将决策推迟到人类在更可能比机器学习分类器更准确时。现有的 L2D 研究忽视了阻碍其实际应用的现实系统的关键要素，即：i）忽略了具有不同成本的类型 1 错误和类型 2 错误的成本敏感场景；ii）要求对训练数据集的每个实例进行并发的人类预测；iii）不处理人类工作能力的限制。为了解决这些问题，我们提出了成本和能力约束下的推迟框架（DeCCaF）。DeCCaF 是一种新的 L2D 方法，采用监督学习来建模人类错误的概率，满足更宽松的数据要求（每个实例只有一个专家预测），并使用约束编程来在工作负载限制下全局最小化错误成本。我们在一系列具有不同 9 名合成欺诈分析师团队和个体工作能力限制的成本敏感型欺诈检测场景中测试了 DeCCaF。结果表明，我们的方法在各种场景中比基准方案表现显著好，平均减少了 8.4% 的错误分类成本。