This paper proposes a novel scalable reinforcement learning approach for
simultaneous routing and spectrum access in wireless ad-hoc networks. In most
previous works on reinforcement learning for network optimization, the network
topology is assumed to be fixed, and a different agent is trained for each
transmission node -- this limits scalability and generalizability. Further,
routing and spectrum access are typically treated as separate tasks. Moreover,
the optimization objective is usually a cumulative metric along the route,
e.g., number of hops or delay. In this paper, we account for the physical-layer
signal-to-interference-plus-noise ratio (SINR) in a wireless network and
further show that bottleneck objective such as the minimum SINR along the route
can also be optimized effectively using reinforcement learning. Specifically,
we propose a scalable approach in which a single agent is associated with each
flow and makes routing and spectrum access decisions as it moves along the
frontier nodes. The agent is trained according to the physical-layer
characteristics of the environment using a novel rewarding scheme based on the
Monte Carlo estimation of the future bottleneck SINR. It learns to avoid
interference by intelligently making joint routing and spectrum allocation
decisions based on the geographical location information of the neighbouring
nodes.

本文提出了一种可扩展的强化学习方法，用于同时路由和频谱访问无线自组网中的信号传输，根据邻居节点的地理位置信息智能地制定联合路由和频谱分配决策。