Intention is an important and challenging concept in AI. It is important
because it underlies many other concepts we care about, such as agency,
manipulation, legal responsibility, and blame. However, ascribing intent to AI
systems is contentious, and there is no universally accepted theory of
intention applicable to AI agents. We operationalise the intention with which
an agent acts, relating to the reasons it chooses its decision. We introduce a
formal definition of intention in structural causal influence models, grounded
in the philosophy literature on intent and applicable to real-world machine
learning systems. Through a number of examples and results, we show that our
definition captures the intuitive notion of intent and satisfies desiderata
set-out by past work. In addition, we show how our definition relates to past
concepts, including actual causality, and the notion of instrumental goals,
which is a core idea in the literature on safe AI agents. Finally, we
demonstrate how our definition can be used to infer the intentions of
reinforcement learning agents and language models from their behaviour.

通过在结构因果影响模型中引入意图的形式定义，我们展示了这个定义捕捉到意图的直观概念，并满足了过去工作设定的期望。此外，我们还展示了我们的定义如何与过去的概念相联系，包括实际因果关系和安全 AI 代理文献中的核心概念 —— 工具性目标。最后，我们演示了如何从行为中推断强化学习代理和语言模型的意图。