A multi-agent deep reinforcement learning (MADRL) is a promising approach to
challenging problems in wireless environments involving multiple
decision-makers (or actors) with high-dimensional continuous action space. In
this paper, we present a MADRL-based approach that can jointly optimize
precoders to achieve the outer-boundary, called pareto-boundary, of the
achievable rate region for a multiple-input single-output (MISO) interference
channel (IFC). In order to address two main challenges, namely, multiple actors
(or agents) with partial observability and multi-dimensional continuous action
space in MISO IFC setup, we adopt a multi-agent deep deterministic policy
gradient (MA-DDPG) framework in which decentralized actors with partial
observability can learn a multi-dimensional continuous policy in a centralized
manner with the aid of shared critic with global information. Meanwhile, we
will also address a phase ambiguity issue with the conventional complex
baseband representation of signals widely used in radio communications. In
order to mitigate the impact of phase ambiguity on training performance, we
propose a training method, called phase ambiguity elimination (PAE), that leads
to faster learning and better performance of MA-DDPG in wireless communication
systems. The simulation results exhibit that MA-DDPG is capable of learning a
near-optimal precoding strategy in a MISO IFC environment. To the best of our
knowledge, this is the first work to demonstrate that the MA-DDPG framework can
jointly optimize precoders to achieve the pareto-boundary of achievable rate
region in a multi-cell multi-user multi-antenna system.

本文提出了一种 MADRL 的方法，即采用 MA-DDPG 框架，在有多个决策制定者并且这些决策制定者只能观测到部分环境信息，且存在多维度动作空间的情况下，在多输入单输出干扰信道系统中联合优化预编码器，来实现可实现速率区域的边界，同时提出了一种解决相位歧义问题的训练方法（PAE），模拟结果表明这种方法在 MISO IFC 系统中可以学习到近乎最优的预编码策略，并且这是第一篇在多个蜂窝、多用户、多天线系统中证明 MA-DDPG 框架可以联合优化预编码器以达到可实现速率区域的边界的研究。

多智能体深度强化学习（MADRL）遇见多用户 MIMO 系统

Multi-agent deep reinforcement learning (MADRL) meets multi-user MIMO  systems

Reinforcement learning (RL) algorithms have been around for decades and
employed to solve various sequential decision-making problems. These algorithms
however have faced great challenges when dealing with high-dimensional
environments. The recent development of deep learning has enabled RL methods to
drive optimal policies for sophisticated and capable agents, which can perform
efficiently in these challenging environments. This paper addresses an
important aspect of deep RL related to situations that require multiple agents
to communicate and cooperate to solve complex tasks. A survey of different
approaches to problems related to multi-agent deep RL (MADRL) is presented,
including non-stationarity, partial observability, continuous state and action
spaces, multi-agent training schemes, multi-agent transfer learning. The merits
and demerits of the reviewed methods will be analyzed and discussed, with their
corresponding applications explored. It is envisaged that this review provides
insights about various MADRL methods and can lead to future development of more
robust and highly useful multi-agent learning methods for solving real-world
problems.

此篇论文介绍了多智能体深度强化学习的不同方法，包括非静态性、部分可观测性、连续的状态和操作空间、多智能体训练机制、多智能体转移学习，并分析和讨论了这些方法的优缺点及其相关应用，旨在促进更加健壮和高效的多智能体学习方法的发展。