Cellular-connected unmanned aerial vehicle (UAV) is a promising technology to
unlock the full potential of UAVs in the future. However, how to achieve
ubiquitous three-dimensional (3D) communication coverage for the UAVs in the
sky is a new challenge. In this paper, we tackle this challenge by a new
coverage-aware navigation approach, which exploits the UAV's controllable
mobility to design its navigation/trajectory to avoid the cellular BSs'
coverage holes while accomplishing their missions. We formulate an UAV
trajectory optimization problem to minimize the weighted sum of its mission
completion time and expected communication outage duration, and propose a new
solution approach based on the technique of deep reinforcement learning (DRL).
To further improve the performance, we propose a new framework called
simultaneous navigation and radio mapping (SNARM), where the UAV's signal
measurement is used not only for training the deep Q network (DQN) directly,
but also to create a radio map that is able to predict the outage probabilities
at all locations in the area of interest. This thus enables the generation of
simulated UAV trajectories and predicting their expected returns, which are
then used to further train the DQN via Dyna technique, thus greatly improving
the learning efficiency.

本文提出了一种基于深度强化学习的覆盖感知导航方法，该方法利用移动无人机的可控机动性设计其导航 / 轨迹，避免了蜂窝基站覆盖漏洞，并提出了一种同时导航和无线电映射的框架。该模型通过深度 Q 网络（DQN）预测位置的中断概率，并预测期望回报，从而使得机器人学习效率大幅提升。