Reinforcement learning (RL) has been a promising essence in future 5G-beyond and 6G systems. Its main advantage lies in its robust model-free decision-making in complex and large-dimension wireless environments. However, most existing RL frameworks rely on online interaction with the environment, which might not be feasible due to safety and cost concerns. Another problem with online RL is the lack of scalability of the designed algorithm with dynamic or new environments. This work proposes a novel, resilient, few-shot meta-offline RL algorithm combining offline RL using conservative Q-learning (CQL) and meta-learning using model-agnostic meta-learning (MAML). The proposed algorithm can train RL models using static offline datasets without any online interaction with the environments. In addition, with the aid of MAML, the proposed model can be scaled up to new unseen environments. We showcase the proposed algorithm for optimizing an unmanned aerial vehicle (UAV) 's trajectory and scheduling policy to minimize the age-of-information (AoI) and transmission power of limited-power devices. Numerical results show that the proposed few-shot meta-offline RL algorithm converges faster than baseline schemes, such as deep Q-networks and CQL. In addition, it is the only algorithm that can achieve optimal joint AoI and transmission power using an offline dataset with few shots of data points and is resilient to network failures due to unprecedented environmental changes.

本研究解决了现有强化学习框架在安全性和成本问题上对在线交互的依赖，以及在动态或新环境中的可扩展性不足。提出了一种新颖的少样本元离线强化学习算法，能够在没有在线交互的情况下，利用静态离线数据集训练模型，并实现对新环境的有效扩展。研究结果表明，该算法在优化无人机轨迹和调度策略方面表现优越，能够快速收敛并在数据点少的情况下实现最佳的年龄信息和传输功率平衡。

通过少样本元离线强化学习实现鲁棒的无人机轨迹规划