We study the data packet transmission problem (mmDPT) in dense cell-free
millimeter wave (mmWave) networks, i.e., users sending data packet requests to
access points (APs) via uplinks and APs transmitting requested data packets to
users via downlinks. Our objective is to minimize the average delay in the
system due to APs' limited service capacity and unreliable wireless channels
between APs and users. This problem can be formulated as a restless multi-armed
bandits problem with fairness constraint (RMAB-F). Since finding the optimal
policy for RMAB-F is intractable, existing learning algorithms are
computationally expensive and not suitable for practical dynamic dense mmWave
networks. In this paper, we propose a structured reinforcement learning (RL)
solution for mmDPT by exploiting the inherent structure encoded in RMAB-F. To
achieve this, we first design a low-complexity and provably asymptotically
optimal index policy for RMAB-F. Then, we leverage this structure information
to develop a structured RL algorithm called mmDPT-TS, which provably achieves
an \tilde{O}(\sqrt{T}) Bayesian regret. More importantly, mmDPT-TS is
computation-efficient and thus amenable to practical implementation, as it
fully exploits the structure of index policy for making decisions. Extensive
emulation based on data collected in realistic mmWave networks demonstrate
significant gains of mmDPT-TS over existing approaches.

通过利用编码在 RMAB-F 中的内在结构，我们提出了一个结构化强化学习解决方案 mmDPT-TS，最小化了由 AP 的有限服务能力和 AP 与用户之间不可靠无线信道引起的系统平均延迟。

密集毫米波网络中延迟优化数据传输的结构化强化学习

Structured Reinforcement Learning for Delay-Optimal Data Transmission in  Dense mmWave Networks

In this paper, a novel framework is proposed for channel charting (CC)-aided
localization in millimeter wave networks. In particular, a convolutional
autoencoder model is proposed to estimate the three-dimensional location of
wireless user equipment (UE), based on multipath channel state information
(CSI), received by different base stations. In order to learn the
radio-geometry map and capture the relative position of each UE, an
autoencoder-based channel chart is constructed in an unsupervised manner, such
that neighboring UEs in the physical space will remain close in the channel
chart. Next, the channel charting model is extended to a semi-supervised
framework, where the autoencoder is divided into two components: an encoder and
a decoder, and each component is optimized individually, using the labeled CSI
dataset with associated location information, to further improve positioning
accuracy. Simulation results show that the proposed CC-aided semi-supervised
localization yields a higher accuracy, compared with existing supervised
positioning and conventional unsupervised CC approaches.

本文提出了一种基于卷积自编码器的信道图辅助定位方法，结合多路径信道状态信息，通过无人监督学习构建信道图以捕捉每个无线设备的相对位置，再引入半监督学习，进一步提高了定位精度。与现有的有监督和传统无监督方法相比，所提出的方法在精度上得到了显著提高。