This article presents a digital twin (DT)-enhanced reinforcement learning
(RL) framework aimed at optimizing performance and reliability in network
resource management, since the traditional RL methods face several unified
challenges when applied to physical networks, including limited exploration
efficiency, slow convergence, poor long-term performance, and safety concerns
during the exploration phase. To deal with the above challenges, a
comprehensive DT-based framework is proposed to enhance the convergence speed
and performance for unified RL-based resource management. The proposed
framework provides safe action exploration, more accurate estimates of
long-term returns, faster training convergence, higher convergence performance,
and real-time adaptation to varying network conditions. Then, two case studies
on ultra-reliable and low-latency communication (URLLC) services and multiple
unmanned aerial vehicles (UAV) network are presented, demonstrating
improvements of the proposed framework in performance, convergence speed, and
training cost reduction both on traditional RL and neural network based Deep RL
(DRL). Finally, the article identifies and explores some of the research
challenges and open issues in this rapidly evolving field.

提出了一种基于数字孪生增强的强化学习框架，旨在优化网络资源管理的性能和可靠性，此框架在物理网络中应用传统的强化学习面临多个统一挑战，包括有限的探索效率、收敛速度缓慢、长期性能较差以及在探索阶段的安全问题。

通过数字孪生实现增强式强化学习资源管理：机遇、应用和挑战

Toward Enhanced Reinforcement Learning-Based Resource Management via  Digital Twin: Opportunities, Applications, and Challenges

The emerging technology of multi-tenancy network slicing is considered as an
essential feature of 5G cellular networks. It provides network slices as a new
type of public cloud services, and therewith increases the service flexibility
and enhances the network resource efficiency. Meanwhile, it raises new
challenges of network resource management. A number of various methods have
been proposed over the recent past years, in which machine learning and
artificial intelligence techniques are widely deployed. In this article, we
provide a survey to existing approaches of network slicing resource management,
with a highlight on the roles played by machine learning in them.

本文综述了多租户网络分片技术在 5G 蜂窝网络中的重要性，其提供了作为一种新类型公共云服务的网络切片，因此增加了服务的灵活性和增强了网络资源效率，并探讨了机器学习在其中的角色。