The Iterated Prisoner's Dilemma has guided research on social dilemmas for
decades. However, it distinguishes between only two atomic actions: cooperate
and defect. In real-world prisoner's dilemmas, these choices are temporally
extended and different strategies may correspond to sequences of actions,
reflecting grades of cooperation. We introduce a Sequential Prisoner's Dilemma
(SPD) game to better capture the aforementioned characteristics. In this work,
we propose a deep multiagent reinforcement learning approach that investigates
the evolution of mutual cooperation in SPD games. Our approach consists of two
phases. The first phase is offline: it synthesizes policies with different
cooperation degrees and then trains a cooperation degree detection network. The
second phase is online: an agent adaptively selects its policy based on the
detected degree of opponent cooperation. The effectiveness of our approach is
demonstrated in two representative SPD 2D games: the Apple-Pear game and the
Fruit Gathering game. Experimental results show that our strategy can avoid
being exploited by exploitative opponents and achieve cooperation with
cooperative opponents.

通过提出一种序列囚徒困境游戏和使用深度多智能体强化学习方法进行研究，本文探讨互相合作演化趋势。实验表明，该策略可避免被攻击并与合作的对手实现合作。