Multi-agent reinforcement learning (MARL) has exploded in popularity in recent years. Many approaches have been developed but they can be divided into three main types: centralized training and execution (CTE), centralized training for decentralized execution (CTDE), and Decentralized training and execution (DTE). CTDE methods are the most common as they can use centralized information during training but execute in a decentralized manner -- using only information available to that agent during execution. CTDE is the only paradigm that requires a separate training phase where any available information (e.g., other agent policies, underlying states) can be used. As a result, they can be more scalable than CTE methods, do not require communication during execution, and can often perform well. CTDE fits most naturally with the cooperative case, but can be potentially applied in competitive or mixed settings depending on what information is assumed to be observed. This text is an introduction to CTDE in cooperative MARL. It is meant to explain the setting, basic concepts, and common methods. It does not cover all work in CTDE MARL as the subarea is quite extensive. I have included work that I believe is important for understanding the main concepts in the subarea and apologize to those that I have omitted.

本文研究了合作多智能体强化学习中的集中训练与分散执行（CTDE）方法，解决了现有方法在执行时需依赖本地信息的问题。通过介绍CTDE的基本概念和常用方法，强调其在合作场景中的自然适用性，本文的核心发现是CTDE可以在不需要执行阶段通信的情况下，提升智能体的可扩展性和性能。

集中训练与分散执行在合作多智能体强化学习中的介绍