As artificial intelligence (AI) models continue to scale up, they are
becoming more capable and integrated into various forms of decision-making
systems. For models involved in moral decision-making, also known as artificial
moral agents (AMA), interpretability provides a way to trust and understand the
agent's internal reasoning mechanisms for effective use and error correction.
In this paper, we provide an overview of this rapidly-evolving sub-field of AI
interpretability, introduce the concept of the Minimum Level of
Interpretability (MLI) and recommend an MLI for various types of agents, to aid
their safe deployment in real-world settings.

本文概述了人工智能可解释性的一个快速发展子领域 - 基于道德决策的可解释性模型，并介绍了最小可解释性水平 (Minimum Level of Interpretability, MLI) 的概念，以及为不同类型的代理建议 MLI 的安全部署方式。

人工道德代理的最低可解释性要求

Minimum Levels of Interpretability for Artificial Moral Agents

To achieve desirable performance, current AI systems often require huge
amounts of training data. This is especially problematic in domains where
collecting data is both expensive and time-consuming, e.g., where AI systems
require having numerous interactions with humans, collecting feedback from
them. In this work, we substantiate the idea of $\textit{cognitive models as
simulators}$, which is to have AI systems interact with, and collect feedback
from, cognitive models instead of humans, thereby making their training process
both less costly and faster. Here, we leverage this idea in the context of
moral decision-making, by having reinforcement learning (RL) agents learn about
fairness through interacting with a cognitive model of the Ultimatum Game (UG),
a canonical task in behavioral and brain sciences for studying fairness.
Interestingly, these RL agents learn to rationally adapt their behavior
depending on the emotional state of their simulated UG responder. Our work
suggests that using cognitive models as simulators of humans is an effective
approach for training AI systems, presenting an important way for computational
cognitive science to make contributions to AI.

该研究提出了使用认知模型作为模拟人类代替人类与人工智能交互和收集反馈以提高训练效率的方法，并在道德决策制定方面实践，表明这是计算认知科学对人工智能的重要贡献。其中使用强化学习代理与认知模型交互学习公平性，并能理性调整行为，以此为例说明认知模型作为人类模拟器在训练人工智能系统方面是有效的。