In this paper, the problem of using one active unmanned aerial vehicle (UAV)
and four passive UAVs to localize a 3D target UAV in real time is investigated.
In the considered model, each passive UAV receives reflection signals from the
target UAV, which are initially transmitted by the active UAV. The received
reflection signals allow each passive UAV to estimate the signal transmission
distance which will be transmitted to a base station (BS) for the estimation of
the position of the target UAV. Due to the movement of the target UAV, each
active/passive UAV must optimize its trajectory to continuously localize the
target UAV. Meanwhile, since the accuracy of the distance estimation depends on
the signal-to-noise ratio of the transmission signals, the active UAV must
optimize its transmit power. This problem is formulated as an optimization
problem whose goal is to jointly optimize the transmit power of the active UAV
and trajectories of both active and passive UAVs so as to maximize the target
UAV positioning accuracy. To solve this problem, a Z function decomposition
based reinforcement learning (ZD-RL) method is proposed. Compared to value
function decomposition based RL (VD-RL), the proposed method can find the
probability distribution of the sum of future rewards to accurately estimate
the expected value of the sum of future rewards thus finding better transmit
power of the active UAV and trajectories for both active and passive UAVs and
improving target UAV positioning accuracy. Simulation results show that the
proposed ZD-RL method can reduce the positioning errors by up to 39.4% and
64.6%, compared to VD-RL and independent deep RL methods, respectively.

本文研究使用一个主动无人机和四个被动无人机实时定位一个三维目标无人机的问题，并提出了基于 Z 函数分解的强化学习方法，通过优化无人机的轨迹和主动无人机的发送功率，以最大化目标无人机的定位精度。仿真结果表明，所提出的 ZD-RL 方法相对于 VD-RL 和独立深度 RL 方法，可以减小定位误差多达 39.4% 和 64.6%。