Multicast communication technology is widely applied in wireless environments
with a high device density. Traditional wireless network architectures have
difficulty flexibly obtaining and maintaining global network state information
and cannot quickly respond to network state changes, thus affecting the
throughput, delay, and other QoS requirements of existing multicasting
solutions. Therefore, this paper proposes a new multicast routing method based
on multiagent deep reinforcement learning (MADRL-MR) in a software-defined
wireless networking (SDWN) environment. First, SDWN technology is adopted to
flexibly configure the network and obtain network state information in the form
of traffic matrices representing global network links information, such as link
bandwidth, delay, and packet loss rate. Second, the multicast routing problem
is divided into multiple subproblems, which are solved through multiagent
cooperation. To enable each agent to accurately understand the current network
state and the status of multicast tree construction, the state space of each
agent is designed based on the traffic and multicast tree status matrices, and
the set of AP nodes in the network is used as the action space. A novel
single-hop action strategy is designed, along with a reward function based on
the four states that may occur during tree construction: progress, invalid,
loop, and termination. Finally, a decentralized training approach is combined
with transfer learning to enable each agent to quickly adapt to dynamic network
changes and accelerate convergence. Simulation experiments show that MADRL-MR
outperforms existing algorithms in terms of throughput, delay, packet loss
rate, etc., and can establish more intelligent multicast routes.

本文提出了一种基于多智能体深度强化学习的新型组播路由方法，该方法应用于软件定义的无线网络环境中，灵活配置网络并获得网络状态信息。使用单跳动作策略和奖励函数解决多智能体合作下的多个子问题，并采用分散式训练方法相结合的迁移学习机制来加速收敛并提高吞吐量、延迟、丢包率等性能，模拟实验表明 MADRL-MR 方法可以建立更智能的组播路由。