The model-based power allocation algorithm has been investigated for decades,
but it requires the mathematical models to be analytically tractable and it
usually has high computational complexity. Recently, the data-driven model-free
machine learning enabled approaches are being rapidly developed to obtain
near-optimal performance with affordable computational complexity, and deep
reinforcement learning (DRL) is regarded as of great potential for future
intelligent networks. In this paper, the DRL approaches are considered for
power control in multi-user wireless communication cellular networks.
Considering the cross-cell cooperation, the off-line/on-line centralized
training and the distributed execution, we present a mathematical analysis for
the DRL-based top-level design. The concrete DRL design is further developed
based on this foundation, and policy-based REINFORCE, value-based deep Q
learning (DQL), actor-critic deep deterministic policy gradient (DDPG)
algorithms are proposed. Simulation results show that the proposed data-driven
approaches outperform the state-of-art model-based methods on sum-rate
performance, with good generalization power and faster processing speed.
Furthermore, the proposed DDPG outperforms the REINFORCE and DQL in terms of
both sum-rate performance and robustness, and can be incorporated into existing
resource allocation schemes due to its generality.

本文研究了基于深度强化学习的无模型无需解析解的动力控制方案在跨单元合作、离线 / 在线集中训练和分布式执行等方面的数学分析和具体实现。 分析和仿真结果表明，DRL 设计在性能、鲁棒性和广泛可用性方面优于基于模型的方法，特别是 actor-critic 深度确定性策略梯度算法，可用于现有资源分配方案。