This paper studies the path design problem for cellular-connected unmanned
aerial vehicle (UAV), which aims to minimize its mission completion time while
maintaining good connectivity with the cellular network. We first argue that
the conventional path design approach via formulating and solving optimization
problems faces several practical challenges, and then propose a new
reinforcement learning-based UAV path design algorithm by applying
\emph{temporal-difference} method to directly learn the \emph{state-value
function} of the corresponding Markov Decision Process. The proposed algorithm
is further extended by using linear function approximation with tile coding to
deal with large state space. The proposed algorithms only require the raw
measured or simulation-generated signal strength as the input and are suitable
for both online and offline implementations. Numerical results show that the
proposed path designs can successfully avoid the coverage holes of cellular
networks even in the complex urban environment.

本文研究了无人机的路径设计问题，提出了一种新的基于强化学习的算法，通过学习相应的 MDP 的状态 - 价值函数来求解，并使用瓦片编码对大状态空间进行处理。该算法使用原始测量或仿真生成的信号强度作为输入，可适用于在线和离线实现，并成功地避免了城市环境中蜂窝网络的覆盖漏洞。