This paper proposes a novel scalable reinforcement learning approach for
simultaneous routing and spectrum access in wireless ad-hoc networks. In most
previous works on reinforcement learning for network optimization, the network
topology is assumed to be fixed, and a different agent is trained for each
transmission node -- this limits scalability and generalizability. Further,
routing and spectrum access are typically treated as separate tasks. Moreover,
the optimization objective is usually a cumulative metric along the route,
e.g., number of hops or delay. In this paper, we account for the physical-layer
signal-to-interference-plus-noise ratio (SINR) in a wireless network and
further show that bottleneck objective such as the minimum SINR along the route
can also be optimized effectively using reinforcement learning. Specifically,
we propose a scalable approach in which a single agent is associated with each
flow and makes routing and spectrum access decisions as it moves along the
frontier nodes. The agent is trained according to the physical-layer
characteristics of the environment using a novel rewarding scheme based on the
Monte Carlo estimation of the future bottleneck SINR. It learns to avoid
interference by intelligently making joint routing and spectrum allocation
decisions based on the geographical location information of the neighbouring
nodes.

本文提出了一种可扩展的强化学习方法，用于同时路由和频谱访问无线自组网中的信号传输，根据邻居节点的地理位置信息智能地制定联合路由和频谱分配决策。

物理层路由和频谱访问的可扩展深度强化学习

Scalable Deep Reinforcement Learning for Routing and Spectrum Access in  Physical Layer

Cognitive radio networks (CRNs) have emerged as an essential technology to
enable dynamic and opportunistic spectrum access which aims to exploit
underutilized licensed channels to solve the spectrum scarcity problem. Despite
the great benefits that CRNs offer in terms of their ability to improve
spectrum utilization efficiency, they suffer from user location privacy issues.
Knowing that their whereabouts may be exposed can discourage users from joining
and participating in the CRNs, thereby potentially hindering the adoption and
deployment of this technology in future generation networks. The location
information leakage issue in the CRN context has recently started to gain
attention from the research community due to its importance, and several
research efforts have been made to tackle it. However, to the best of our
knowledge, none of these works have tried to identify the vulnerabilities that
are behind this issue or discuss the approaches that could be deployed to
prevent it. In this paper, we try to fill this gap by providing a comprehensive
survey that investigates the various location privacy risks and threats that
may arise from the different components of this CRN technology, and explores
the different privacy attacks and countermeasure solutions that have been
proposed in the literature to cope with this location privacy issue. We also
discuss some open research problems, related to this issue, that need to be
overcome by the research community to take advantage of the benefits of this
key CRN technology without having to sacrifice the users' privacy.

本文探讨了认知无线电网络中与位置隐私相关的各种潜在威胁和风险，同时探讨了文献中提出的隐私攻击和相应的解决方案，为解决该问题提供了全面的调查结果。