Fairness in AI is a growing concern for high-stakes decision making. Engaging
stakeholders, especially lay users, in fair AI development is promising yet
overlooked. Recent efforts explore enabling lay users to provide AI
fairness-related feedback, but there is still a lack of understanding of how to
integrate users' feedback into an AI model and the impacts of doing so. To
bridge this gap, we collected feedback from 58 lay users on the fairness of a
XGBoost model trained on the Home Credit dataset, and conducted offline
experiments to investigate the effects of retraining models on accuracy, and
individual and group fairness. Our work contributes baseline results of
integrating user fairness feedback in XGBoost, and a dataset and code framework
to bootstrap research in engaging stakeholders in AI fairness. Our discussion
highlights the challenges of employing user feedback in AI fairness and points
the way to a future application area of interactive machine learning.

人工智能的公平性在高风险决策中越来越引起关注，让利益相关者，尤其是普通用户，参与公平的人工智能开发具有潜力但往往被忽视。最近的努力探索了让普通用户提供与公平性相关的反馈的方法，但对如何将用户的反馈整合到人工智能模型中以及这样做的影响尚不了解。为了填补这一差距，我们从 58 个普通用户收集了关于一个在 Home Credit 数据集上训练的 XGBoost 模型公平性的反馈，并进行了离线实验，以研究重新训练模型对准确性、个体和团体公平性的影响。我们的工作为在 XGBoost 中整合用户公平反馈提供了基准结果，并提供了一个数据集和代码框架，以推动参与利益相关者研究人工智能公平性的起步工作。我们的讨论突出了在人工智能公平性中使用用户反馈面临的挑战，并指出了交互式机器学习的未来应用领域的方向。