Algorithmic systems -- from rule-based bots to machine learning classifiers -- have a long history of supporting the essential work of content moderation and other curation work in peer production projects. From counter-vandalism to task routing, basic machine prediction has allowed open knowledge projects like Wikipedia to scale to the largest encyclopedia in the world, while maintaining quality and consistency. However, conversations about how quality control should work and what role algorithms should play have generally been led by the expert engineers who have the skills and resources to develop and modify these complex algorithmic systems. In this paper, we describe ORES: an algorithmic scoring service that supports real-time scoring of wiki edits using multiple independent classifiers trained on different datasets. ORES decouples several activities that have typically all been performed by engineers: choosing or curating training data, building models to serve predictions, auditing predictions, and developing interfaces or automated agents that act on those predictions. This meta-algorithmic system was designed to open up socio-technical conversations about algorithmic systems in Wikipedia to a broader set of participants. In this paper, we discuss the theoretical mechanisms of social change ORES enables and detail case studies in participatory machine learning around ORES from the 4 years since its deployment.

本文介绍了ORES：一种算法评分服务，支持在不同数据集上训练的多个独立分类器对wiki编辑进行实时评分，从而解耦了通常由工程师执行的许多活动。ORES的设计旨在将关于维基百科算法的社会技术对话开放给更广泛的参与者。文中还讨论了ORES启用的社会变革机制的理论机制，并详细介绍了自其部署以来5年来围绕ORES的参与式机器学习案例研究。

ORES:在维基百科中使用参与式机器学习降低门槛