The deployment of AI systems for welfare benefit allocation allows for
accelerated decision-making and faster provision of critical help, but has
already led to an increase in unfair benefit denials and false fraud
accusations. Collecting data in the US and the UK (N = 2449), we explore the
public acceptability of such speed-accuracy trade-offs in populations of
claimants and non-claimants. We observe a general willingness to trade off
speed gains for modest accuracy losses, but this aggregate view masks notable
divergences between claimants and non-claimants. Although welfare claimants
comprise a relatively small proportion of the general population (e.g., 20% in
the US representative sample), this vulnerable group is much less willing to
accept AI deployed in welfare systems, raising concerns that solely using
aggregate data for calibration could lead to policies misaligned with
stakeholder preferences. Our study further uncovers asymmetric insights between
claimants and non-claimants. The latter consistently overestimate claimant
willingness to accept speed-accuracy trade-offs, even when financially
incentivized for accurate perspective-taking. This suggests that policy
decisions influenced by the dominant voice of non-claimants, however
well-intentioned, may neglect the actual preferences of those directly affected
by welfare AI systems. Our findings underline the need for stakeholder
engagement and transparent communication in the design and deployment of these
systems, particularly in contexts marked by power imbalances.

AI 系统的部署用于福利补贴分配，加速决策和提供关键帮助，但已导致不公平的拒绝福利和虚假欺诈指控的增加。通过在美国和英国收集数据（N = 2449），我们探讨了此类速度 - 准确性权衡在索赔人群和非索赔人群的公众可接受性。我们观察到人们普遍愿意以适度的准确性损失为代价来换取速度的提高，但这种总体观点掩盖了索赔人群和非索赔人群之间的显著差异。索赔人群对于在福利系统中部署 AI 的接受度较低，尽管福利索赔人群在整个人群中占比相对较小（例如，在美国代表性样本中为 20%），这一弱势群体不太愿意接受 AI，这引发了使用聚合数据用于校准可能导致与利益相关方偏好不一致的政策的担忧。我们的研究还发现了索赔人群和非索赔人群之间的非对称洞察。即使在为准确的透视视角提供经济激励的情况下，后者始终高估索赔人群接受速度 - 准确性权衡的意愿。这表明受非索赔人群主导的政策决策，无论出于何种善意，都可能忽视那些直接受到福利 AI 系统影响的人的实际偏好。我们的研究结果强调了在设计和部署这些系统时需要利益相关方参与和透明沟通的重要性，特别是在存在权力失衡的环境中。

虚假共识偏见导致人工智能对弱势利益相关者的不公

False consensus biases AI against vulnerable stakeholders

Emerging methods for participatory algorithm design have proposed collecting
and aggregating individual stakeholder preferences to create algorithmic
systems that account for those stakeholders' values. Using algorithmic student
assignment as a case study, we argue that optimizing for individual preference
satisfaction in the distribution of limited resources may actually inhibit
progress towards social and distributive justice. Individual preferences can be
a useful signal but should be expanded to support more expressive and inclusive
forms of democratic participation.

参与式算法设计越来越普遍，通过收集并汇总利益相关者的个人偏好，可以创建算法系统，考虑到这些利益相关者的价值观。使用算法学生分配作为案例研究，我们认为，在限制资源的分配中优化个人偏好满足可能会抑制社会和分配的公正进展。个人偏好可以是有用的信号，但应该扩展以支持更具表现力和包容性的民主参与形式。