Network slicing-based communication systems can dynamically and efficiently
allocate resources for diversified services. However, due to the limitation of
the network interface on channel access and the complexity of the resource
allocation, it is challenging to achieve an acceptable solution in the
practical system without precise prior knowledge of the dynamics probability
model of the service requests. Existing work attempts to solve this problem
using deep reinforcement learning (DRL), however, such methods usually require
a lot of interaction with the real environment in order to achieve good
results. In this paper, a framework consisting of a digital twin and
reinforcement learning agents is present to handle the issue. Specifically, we
propose to use the historical data and the neural networks to build a digital
twin model to simulate the state variation law of the real environment. Then,
we use the data generated by the network slicing environment to calibrate the
digital twin so that it is in sync with the real environment. Finally, DRL for
slice optimization optimizes its own performance in this virtual
pre-verification environment. We conducted an exhaustive verification of the
proposed digital twin framework to confirm its scalability. Specifically, we
propose to use loss landscapes to visualize the generalization of DRL
solutions. We explore a distillation-based optimization scheme for lightweight
slicing strategies. In addition, we also extend the framework to offline
reinforcement learning, where solutions can be used to obtain intelligent
decisions based solely on historical data. Numerical simulation experiments
show that the proposed digital twin can significantly improve the performance
of the slice optimization strategy.

网络切片通信系统通过动态和高效地分配资源来满足各种服务的需求。本文提出了一种由数字孪生和强化学习代理构成的框架来处理资源分配的问题，并验证了该框架的可扩展性，展示了数字孪生对切片优化策略性能的显著改进。