Unmanned aerial vehicles (UAVs) are expected to be a key component of the
next-generation wireless systems. Due to their deployment flexibility, UAVs are
being considered as an efficient solution for collecting information data from
ground nodes and transmitting it wirelessly to the network. In this paper, a
UAV-assisted wireless network is studied, in which energy-constrained ground
nodes are deployed to observe different physical processes. In this network, a
UAV that has a time constraint for its operation due to its limited battery,
moves towards the ground nodes to receive status update packets about their
observed processes. The flight trajectory of the UAV and scheduling of status
update packets are jointly optimized with the objective of achieving the
minimum weighted sum for the age-of-information (AoI) values of different
processes at the UAV, referred to as weighted sum-AoI. The problem is modeled
as a finite-horizon Markov decision process (MDP) with finite state and action
spaces. Since the state space is extremely large, a deep reinforcement learning
(RL) algorithm is proposed to obtain the optimal policy that minimizes the
weighted sum-AoI, referred to as the age-optimal policy. Several simulation
scenarios are considered to showcase the convergence of the proposed deep RL
algorithm. Moreover, the results also demonstrate that the proposed deep RL
approach can significantly improve the achievable sum-AoI per process compared
to the baseline policies, such as the distance-based and random walk policies.
The impact of various system design parameters on the optimal achievable
sum-AoI per process is also shown through extensive simulations.

研究了一种 UAV 辅助的无线网络，通过 Markov 决策过程和深度强化学习算法优化飞行轨迹和任务调度来最小化信息年龄值的加权和，取得了显著的研究成果。