Deep Reinforcement Learning (DRL) has been applied to address a variety of
cooperative multi-agent problems with either discrete action spaces or
continuous action spaces. However, to the best of our knowledge, no previous
work has ever succeeded in applying DRL to multi-agent problems with
discrete-continuous hybrid (or parameterized) action spaces which is very
common in practice. Our work fills this gap by proposing two novel algorithms:
Deep Multi-Agent Parameterized Q-Networks (Deep MAPQN) and Deep Multi-Agent
Hierarchical Hybrid Q-Networks (Deep MAHHQN). We follow the centralized
training but decentralized execution paradigm: different levels of
communication between different agents are used to facilitate the training
process, while each agent executes its policy independently based on local
observations during execution. Our empirical results on several challenging
tasks (simulated RoboCup Soccer and game Ghost Story) show that both Deep MAPQN
and Deep MAHHQN are effective and significantly outperform existing independent
deep parameterized Q-learning method.

本篇论文针对协同多智能体问题中的 “离散 - 连续混合动作空间” 这一实际难题，提出了两个新算法：Deep Multi-Agent Parameterized Q-Networks（Deep MAPQN）和 Deep Multi-Agent Hierarchical Hybrid Q-Networks（Deep MAHHQN）。中心化训练，分散执行范例下的实证结果显示，Deep MAPQN 和 Deep MAHHQN 均更为有效，且远胜于现有独立深度参数化 Q 学习方法。