Discussion of AI alignment (alignment between humans and AI systems) has
focused on value alignment, broadly referring to creating AI systems that share
human values. We argue that before we can even attempt to align values, it is
imperative that AI systems and humans align the concepts they use to understand
the world. We integrate ideas from philosophy, cognitive science, and deep
learning to explain the need for concept alignment, not just value alignment,
between humans and machines. We summarize existing accounts of how humans and
machines currently learn concepts, and we outline opportunities and challenges
in the path towards shared concepts. Finally, we explain how we can leverage
the tools already being developed in cognitive science and AI research to
accelerate progress towards concept alignment.

在讨论人工智能对齐（人与人工智能系统之间的对齐）时，重点放在价值对齐上，广泛指的是创建具有人类价值观的人工智能系统。我们认为，在试图对齐价值之前，AI 系统和人类对于理解世界所使用的概念需要对齐，这是至关重要的。我们结合了哲学、认知科学和深度学习的思想，解释了人类和机器之间需要概念对齐而非仅仅价值对齐的需求。我们总结了人类和机器目前如何学习概念的现有观点，并概述了达成共享概念的机遇和挑战。最后，我们解释了如何利用认知科学和人工智能研究中已经开发的工具来加速实现概念对齐的进展。

概念对齐

Concept Alignment

Value alignment is essential for building AI systems that can safely and
reliably interact with people. However, what a person values -- and is even
capable of valuing -- depends on the concepts that they are currently using to
understand and evaluate what happens in the world. The dependence of values on
concepts means that concept alignment is a prerequisite for value alignment --
agents need to align their representation of a situation with that of humans in
order to successfully align their values. Here, we formally analyze the concept
alignment problem in the inverse reinforcement learning setting, show how
neglecting concept alignment can lead to systematic value mis-alignment, and
describe an approach that helps minimize such failure modes by jointly
reasoning about a person's concepts and values. Additionally, we report
experimental results with human participants showing that humans reason about
the concepts used by an agent when acting intentionally, in line with our joint
reasoning model.

人工智能系统与人进行安全可靠互动所必需的价值对齐依赖于概念对齐，即代理需要与人类相互对齐，以在情境中成功对齐其价值观。本研究通过对逆强化学习环境中的概念对齐问题进行正式分析，并描述了一种有助于减少价值观不对齐的方法，该方法通过同时推理一个人的概念和价值观来最小化这类故障模式。此外，本研究通过人类参与者的实验结果表明，人类在意图行为时会考虑代理使用的概念，符合我们的联合推理模型。