Optimizing edge caching is crucial for the advancement of next-generation
(nextG) wireless networks, ensuring high-speed and low-latency services for
mobile users. Existing data-driven optimization approaches often lack awareness
of the distribution of random data variables and focus solely on optimizing
cache hit rates, neglecting potential reliability concerns, such as base
station overload and unbalanced cache issues. This oversight can result in
system crashes and degraded user experience. To bridge this gap, we introduce a
novel digital twin-assisted optimization framework, called D-REC, which
integrates reinforcement learning (RL) with diverse intervention modules to
ensure reliable caching in nextG wireless networks. We first develop a joint
vertical and horizontal twinning approach to efficiently create network digital
twins, which are then employed by D-REC as RL optimizers and safeguards,
providing ample datasets for training and predictive evaluation of our cache
replacement policy. By incorporating reliability modules into a constrained
Markov decision process, D-REC can adaptively adjust actions, rewards, and
states to comply with advantageous constraints, minimizing the risk of network
failures. Theoretical analysis demonstrates comparable convergence rates
between D-REC and vanilla data-driven methods without compromising caching
performance. Extensive experiments validate that D-REC outperforms conventional
approaches in cache hit rate and load balancing while effectively enforcing
predetermined reliability intervention modules.

为了优化下一代无线网络中的边缘缓存，确保移动用户的高速低延迟服务，本研究引入了一种新颖的数字孪生辅助优化框架 D-REC，它将强化学习与多样化干预模块相结合，以确保可靠的缓存存储，并最小化网络故障的风险。实验结果表明，D-REC 在缓存命中率和负载平衡方面优于传统方法，同时有效地执行预定的可靠性干预模块。

数字孪生辅助数据驱动的无线网络边缘缓存可靠性优化

Digital Twin-Assisted Data-Driven Optimization for Reliable Edge Caching  in Wireless Networks

Content caching at the edge nodes is a promising technique to reduce the data
traffic in next-generation wireless networks. Inspired by the success of Deep
Reinforcement Learning (DRL) in solving complicated control problems, this work
presents a DRL-based framework with Wolpertinger architecture for content
caching at the base station. The proposed framework is aimed at maximizing the
long-term cache hit rate, and it requires no knowledge of the content
popularity distribution. To evaluate the proposed framework, we compare the
performance with other caching algorithms, including Least Recently Used (LRU),
Least Frequently Used (LFU), and First-In First-Out (FIFO) caching strategies.
Meanwhile, since the Wolpertinger architecture can effectively limit the action
space size, we also compare the performance with Deep Q-Network to identify the
impact of dropping a portion of the actions. Our results show that the proposed
framework can achieve improved short-term cache hit rate and improved and
stable long-term cache hit rate in comparison with LRU, LFU, and FIFO schemes.
Additionally, the performance is shown to be competitive in comparison to Deep
Q-learning, while the proposed framework can provide significant savings in
runtime.

使用 DRL 的 Wolpertinger 架构技术实现内容缓存管理，以优化网络缓存命中率，不需要先验知识，与 LRU、LFU、FIFO 策略相比，相应性能更好，且比 DQL 更为稳定和高效。