As AI assistance is increasingly infused into decision-making processes, we
may seek to optimize human-centric objectives beyond decision accuracy, such as
skill improvement or task enjoyment of individuals interacting with these
systems. With this aspiration in mind, we propose offline reinforcement
learning (RL) as a general approach for modeling human-AI decision-making to
optimize such human-centric objectives. Our approach seeks to optimize
different objectives by adaptively providing decision support to humans -- the
right type of assistance, to the right person, at the right time. We
instantiate our approach with two objectives: human-AI accuracy on the
decision-making task and human learning about the task, and learn policies that
optimize these two objectives from previous human-AI interaction data. We
compare the optimized policies against various baselines in AI-assisted
decision-making. Across two experiments (N = 316 and N = 964), our results
consistently demonstrate that people interacting with policies optimized for
accuracy achieve significantly better accuracy -- and even human-AI
complementarity -- compared to those interacting with any other type of AI
support. Our results further indicate that human learning is more difficult to
optimize than accuracy, with participants who interacted with
learning-optimized policies showing significant learning improvement only at
times. Our research (1) demonstrates offline RL to be a promising approach to
model dynamics of human-AI decision-making, leading to policies that may
optimize various human-centric objectives and provide novel insights about the
AI-assisted decision-making space, and (2) emphasizes the importance of
considering human-centric objectives beyond decision accuracy in AI-assisted
decision-making, while also opening up the novel research challenge of
optimizing such objectives.

通过离线强化学习作为一般方法对人工智能决策进行建模，以优化以人为中心的目标，本研究指出人们与经过优化的策略进行交流时，在准确性方面取得了显著提高，并强调了在人工智能决策中考虑除决策准确性之外的人为目标的重要性。

优化 AI 辅助决策中的人本目标与离线强化学习

Towards Optimizing Human-Centric Objectives in AI-Assisted  Decision-Making With Offline Reinforcement Learning

Emerging applications of robotics, and concerns about their impact, require
the research community to put human-centric objectives front-and-center. To
meet this challenge, we advocate an interdisciplinary approach, socially
cognizant robotics, which synthesizes technical and social science methods. We
argue that this approach follows from the need to empower stakeholder
participation (from synchronous human feedback to asynchronous societal
assessment) in shaping AI-driven robot behavior at all levels, and leads to a
range of novel research perspectives and problems both for improving robots'
interactions with individuals and impacts on society. Drawing on these
arguments, we develop best practices for socially cognizant robot design that
balance traditional technology-based metrics (e.g. efficiency, precision and
accuracy) with critically important, albeit challenging to measure, human and
society-based metrics.

构建在技术和社会科学方法的综合基础之上，社会认知机器人是一种跨学科的方法，旨在将人类为中心的目标置于首位，并通过各种研究视角和问题来改进机器人与个体互动以及对社会的影响。