Reciprocity is an important feature of human social interaction and underpins our cooperative nature. What is more, simple forms of reciprocity have proved remarkably resilient in matrix game social dilemmas. Most famously, the tit-for-tat strategy performs very well in tournaments of Prisoner's Dilemma. Unfortunately this strategy is not readily applicable to the real world, in which options to cooperate or defect are temporally and spatially extended. Here, we present a general online reinforcement learning algorithm that displays reciprocal behavior towards its co-players. We show that it can induce pro-social outcomes for the wider group when learning alongside selfish agents, both in a $2$-player Markov game, and in $5$-player intertemporal social dilemmas. We analyse the resulting policies to show that the reciprocating agents are strongly influenced by their co-players' behavior.

本研究提出了一种通用的在线强化学习算法，该算法能够向其合作伙伴表现出互惠行为，并在与自私代理一起学习时能够诱发更广泛的群体互惠行为，无论是在 $2$-player Markov game 还是 $5$-player intertemporal social dilemmas 中。研究分析表明，实施互惠行为的代理受到其合作伙伴行为的强烈影响。

学习复杂连续社交困境中的互惠原则