The global surge in AI applications is transforming industries, leading to
displacement and complementation of existing jobs, while also giving rise to
new employment opportunities. Human oversight of AI is an emerging task in
which human workers interact with an AI model to improve its performance,
safety, and compliance with normative principles. Data annotation, encompassing
the labelling of images or annotating of texts, serves as a critical human
oversight process, as the quality of a dataset directly influences the quality
of AI models trained on it. Therefore, the efficiency of human oversight work
stands as an important competitive advantage for AI developers. This paper
delves into the foundational economics of human oversight, with a specific
focus on the impact of norm design and monetary incentives on data quality and
costs. An experimental study involving 307 data annotators examines six groups
with varying task instructions (norms) and monetary incentives. Results reveal
that annotators provided with clear rules exhibit higher accuracy rates,
outperforming those with vague standards by 14%. Similarly, annotators
receiving an additional monetary incentive perform significantly better, with
the highest accuracy rate recorded in the group working with both clear rules
and incentives (87.5% accuracy). However, both groups require more time to
complete tasks, with a 31% increase in average task completion time compared to
those working with standards and no incentives. These empirical findings
underscore the trade-off between data quality and efficiency in data curation,
shedding light on the nuanced impact of norm design and incentives on the
economics of AI development. The paper contributes experimental insights to
discussions on the economical, ethical, and legal considerations of AI
technologies.

通过实验研究，本文揭示了人工监督、规范设计和激励机制对数据质量和成本的影响，强调数据质量与效率之间的权衡关系，并为讨论人工智能技术的经济、伦理和法律考虑方面提供了实验性见解。

人工智能工作者的成本和绩效受规范和激励影响的经济学

The Economics of Human Oversight: How Norms and Incentives Affect Costs  and Performance of AI Workers

The fast pace of advances in AI promises to revolutionize various aspects of
knowledge work, extending its influence to daily life and professional fields
alike. We advocate for a paradigm where AI is seen as a collaborative co-pilot,
working under human guidance rather than as a mere tool. Drawing from relevant
research and literature in the disciplines of Human-Computer Interaction and
Human Factors Engineering, we highlight the criticality of maintaining human
oversight in AI interactions. Reflecting on lessons from aviation, we address
the dangers of over-relying on automation, such as diminished human vigilance
and skill erosion. Our paper proposes a design approach that emphasizes active
human engagement, control, and skill enhancement in the AI partnership, aiming
to foster a harmonious, effective, and empowering human-AI relationship. We
particularly call out the critical need to design AI interaction capabilities
and software applications to enable and celebrate the primacy of human agency.
This calls for designs for human-AI partnership that cede ultimate control and
responsibility to the human user as pilot, with the AI co-pilot acting in a
well-defined supporting role.

快速发展的人工智能以人工指导为基础的合作副驾驶模式，与仅仅是工具的角色相比，正在改变知识工作的各个方面，并将其影响扩展到日常生活和专业领域。本论文旨在从与人机交互和人因工程学等学科相关的研究和文献中强调在人工智能交互中保持人类监督的重要性，并呼吁通过强调积极的人类参与、控制和技能提升来设计人工智能合作伙伴关系，以促进和谐、有效和赋能的人工智能关系。

AI 副驾驶崛起：来自航空领域及其他领域的设计经验教训

The Rise of the AI Co-Pilot: Lessons for Design from Aviation and Beyond

This article introduces a framework that is meant to assist in mitigating
societal risks that software can pose. Concretely, this encompasses facets of
software doping as well as unfairness and discrimination in high-risk
decision-making systems. The term software doping refers to software that
contains surreptitiously added functionality that is against the interest of
the user. A prominent example of software doping are the tampered emission
cleaning systems that were found in millions of cars around the world when the
diesel emissions scandal surfaced. The first part of this article combines the
formal foundations of software doping analysis with established probabilistic
falsification techniques to arrive at a black-box analysis technique for
identifying undesired effects of software. We apply this technique to emission
cleaning systems in diesel cars but also to high-risk systems that evaluate
humans in a possibly unfair or discriminating way. We demonstrate how our
approach can assist humans-in-the-loop to make better informed and more
responsible decisions. This is to promote effective human oversight, which will
be a central requirement enforced by the European Union's upcoming AI Act. We
complement our technical contribution with a juridically, philosophically, and
psychologically informed perspective on the potential problems caused by such
systems.

本文介绍了一个框架，旨在帮助减轻软件可能带来的社会风险。具体而言，包括软件兴奋剂以及高风险决策系统中的不公平和歧视方面。

人员监督软件掺杂分析

Software Doping Analysis for Human Oversight

The level of autonomy is increasing in systems spanning multiple domains, but
these systems still experience failures. One way to mitigate the risk of
failures is to integrate human oversight of the autonomous systems and rely on
the human to take control when the autonomy fails. In this work, we formulate a
method of collaborative decision making through action suggestions that
improves action selection without taking control of the system. Our approach
uses each suggestion efficiently by incorporating the implicit information
shared through suggestions to modify the agent's belief and achieves better
performance with fewer suggestions than naively following the suggested
actions. We assume collaborative agents share the same objective and
communicate through valid actions. By assuming the suggested action is
dependent only on the state, we can incorporate the suggested action as an
independent observation of the environment. The assumption of a collaborative
environment enables us to use the agent's policy to estimate the distribution
over action suggestions. We propose two methods that use suggested actions and
demonstrate the approach through simulated experiments. The proposed
methodology results in increased performance while also being robust to
suboptimal suggestions.

通过行动建议的协作决策方法，提高自主系统的性能和稳健性。