This paper considers the problem of cost-aware downlink sum-rate maximization
via joint optimal radio access technologies (RATs) assignment and power
allocation in next-generation heterogeneous wireless networks (HetNets). We
consider a future HetNet comprised of multi-RATs and serving multi-connectivity
edge devices (EDs), and we formulate the problem as mixed-integer non-linear
programming (MINP) problem. Due to the high complexity and combinatorial nature
of this problem and the difficulty to solve it using conventional methods, we
propose a hierarchical multi-agent deep reinforcement learning (DRL)-based
framework, called DeepRAT, to solve it efficiently and learn system dynamics.
In particular, the DeepRAT framework decomposes the problem into two main
stages; the RATs-EDs assignment stage, which implements a single-agent Deep Q
Network (DQN) algorithm, and the power allocation stage, which utilizes a
multi-agent Deep Deterministic Policy Gradient (DDPG) algorithm. Using
simulations, we demonstrate how the various DRL agents efficiently interact to
learn system dynamics and derive the global optimal policy. Furthermore, our
simulation results show that the proposed DeepRAT algorithm outperforms
existing state-of-the-art heuristic approaches in terms of network utility.
Finally, we quantitatively show the ability of the DeepRAT model to quickly and
dynamically adapt to abrupt changes in network dynamics, such as EDs mobility.

本文提出了一个基于分阶段多代理深度强化学习框架 DeepRAT 来解决下一代异构无线网络中成本感知的下行总速率最大化的问题。通过模拟实现，我们证明各种 DRL 代理的有效互动，学习系统动态并导出全局最优策略，同时也证明了 DeepRAT 算法在网络效用方面优于现有的最先进的启发式方法，并进一步量化了 DeepRAT 模型在快速适应网络动态、例如设备的移动方面的能力。