The open radio access network (O-RAN) architecture supports intelligent
network control algorithms as one of its core capabilities. Data-driven
applications incorporate such algorithms to optimize radio access network (RAN)
functions via RAN intelligent controllers (RICs). Deep reinforcement learning
(DRL) algorithms are among the main approaches adopted in the O-RAN literature
to solve dynamic radio resource management problems. However, despite the
benefits introduced by the O-RAN RICs, the practical adoption of DRL algorithms
in real network deployments falls behind. This is primarily due to the slow
convergence and unstable performance exhibited by DRL agents upon deployment
and when facing previously unseen network conditions. In this paper, we address
these challenges by proposing transfer learning (TL) as a core component of the
training and deployment workflows for the DRL-based closed-loop control of
O-RAN functionalities. To this end, we propose and design a hybrid TL-aided
approach that leverages the advantages of both policy reuse and distillation TL
methods to provide safe and accelerated convergence in DRL-based O-RAN slicing.
We conduct a thorough experiment that accommodates multiple services, including
real VR gaming traffic to reflect practical scenarios of O-RAN slicing. We also
propose and implement policy reuse and distillation-aided DRL and non-TL-aided
DRL as three separate baselines. The proposed hybrid approach shows at least:
7.7% and 20.7% improvements in the average initial reward value and the
percentage of converged scenarios, and a 64.6% decrease in reward variance
while maintaining fast convergence and enhancing the generalizability compared
with the baselines.

提出了使用深度强化学习 (DRL) 算法的开放无线接入网络 (O-RAN) 切片的转移学习 (TL) 辅助方法，通过政策复用和蒸馏转移学习方法实现了快速收敛和改进泛化能力，显著提高了初始奖励值和收敛场景百分比，减少了奖励方差。