The global surge in AI applications is transforming industries, leading to
displacement and complementation of existing jobs, while also giving rise to
new employment opportunities. Human oversight of AI is an emerging task in
which human workers interact with an AI model to improve its performance,
safety, and compliance with normative principles. Data annotation, encompassing
the labelling of images or annotating of texts, serves as a critical human
oversight process, as the quality of a dataset directly influences the quality
of AI models trained on it. Therefore, the efficiency of human oversight work
stands as an important competitive advantage for AI developers. This paper
delves into the foundational economics of human oversight, with a specific
focus on the impact of norm design and monetary incentives on data quality and
costs. An experimental study involving 307 data annotators examines six groups
with varying task instructions (norms) and monetary incentives. Results reveal
that annotators provided with clear rules exhibit higher accuracy rates,
outperforming those with vague standards by 14%. Similarly, annotators
receiving an additional monetary incentive perform significantly better, with
the highest accuracy rate recorded in the group working with both clear rules
and incentives (87.5% accuracy). However, both groups require more time to
complete tasks, with a 31% increase in average task completion time compared to
those working with standards and no incentives. These empirical findings
underscore the trade-off between data quality and efficiency in data curation,
shedding light on the nuanced impact of norm design and incentives on the
economics of AI development. The paper contributes experimental insights to
discussions on the economical, ethical, and legal considerations of AI
technologies.

通过实验研究，本文揭示了人工监督、规范设计和激励机制对数据质量和成本的影响，强调数据质量与效率之间的权衡关系，并为讨论人工智能技术的经济、伦理和法律考虑方面提供了实验性见解。

人工智能工作者的成本和绩效受规范和激励影响的经济学

The Economics of Human Oversight: How Norms and Incentives Affect Costs  and Performance of AI Workers

We study platforms in the sharing economy and discuss the need for
incentivizing users to explore options that otherwise would not be chosen. For
instance, rental platforms such as Airbnb typically rely on customer reviews to
provide users with relevant information about different options. Yet, often a
large fraction of options does not have any reviews available. Such options are
frequently neglected as viable choices, and in turn are unlikely to be
evaluated, creating a vicious cycle. Platforms can engage users to deviate from
their preferred choice by offering monetary incentives for choosing a different
option instead. To efficiently learn the optimal incentives to offer, we
consider structural information in user preferences and introduce a novel
algorithm - Coordinated Online Learning (CoOL) - for learning with structural
information modeled as convex constraints. We provide formal guarantees on the
performance of our algorithm and test the viability of our approach in a user
study with data of apartments on Airbnb. Our findings suggest that our approach
is well-suited to learn appropriate incentives and increase exploration on the
investigated platform.

通过向用户提供货币奖励来激励他们探索他们可能不会选择的选项是一种有效的方法，使用结构信息模型化为凸约束来学习合适的激励是最佳选择，我们提出了一种新颖的算法 - Coordinated Online Learning（CoOL）用于学习，我们通过测试 Airbnb 上公寓的数据来验证我们的方法是可行的，该方法有助于增加探索。

学习用户偏好以促进共享经济中的探索行为

Learning User Preferences to Incentivize Exploration in the Sharing  Economy

We propose an optimum mechanism for providing monetary incentives to the data
sources of a statistical estimator such as linear regression, so that high
quality data is provided at low cost, in the sense that the sum of payments and
estimation error is minimized. The mechanism applies to a broad range of
estimators, including linear and polynomial regression, kernel regression, and,
under some additional assumptions, ridge regression. It also generalizes to
several objectives, including minimizing estimation error subject to budget
constraints. Besides our concrete results for regression problems, we
contribute a mechanism design framework through which to design and analyze
statistical estimators whose examples are supplied by workers with cost for
labeling said examples.

提出一种最优机制，为统计估算器的数据源提供货币激励，以低成本提供高质量的数据，使支付总额和估算误差最小，广泛应用于线性回归和多项式回归，核回归和岭回归等估算器，并应用于各种目标，包括在预算限制下减少估算误差。