We study the problem of designing AI agents that can learn to cooperate effectively with a potentially suboptimal partner while having no access to the joint reward function. This problem is modeled as a cooperative episodic two-agent Markov decision process. We assume control over only the first of the two agents in a Stackelberg formulation of the game, where the second agent is acting so as to maximise expected utility given the first agent's policy. How should the first agent act in order to learn the joint reward function as quickly as possible, and so that the joint policy is as close to optimal as possible? In this paper, we analyse how knowledge about the reward function can be gained in this interactive two-agent scenario. We show that when the learning agent's policies have a significant effect on the transition function, the reward function can be learned efficiently.

该研究探讨了如何设计自主智能体，使其在没有访问联合奖励函数的情况下能够有效地与潜在的次优合作伙伴进行合作。我们将这个问题建模为一个合作的、情节性的两个代理Markov决策过程。我们分析了该交互式两个代理场景中关于奖励函数的信息如何被获得，结果显示学习代理的策略对转移函数具有显著影响时，奖励函数可以被高效地学习。

合作游戏的交互式逆强化学习