This paper investigates learning-based caching in small-cell networks (SCNs)
when user preference is unknown. The goal is to optimize the cache placement in
each small base station (SBS) for minimizing the system long-term transmission
delay. We model this sequential multi-agent decision making problem in a
multi-agent multi-armed bandit (MAMAB) perspective. Rather than estimating user
preference first and then optimizing the cache strategy, we propose several
MAMAB-based algorithms to directly learn the cache strategy online in both
stationary and non-stationary environment. In the stationary environment, we
first propose two high-complexity agent-based collaborative MAMAB algorithms
with performance guarantee. Then we propose a low-complexity distributed MAMAB
which ignores the SBS coordination. To achieve a better balance between SBS
coordination gain and computational complexity, we develop an edge-based
collaborative MAMAB with the coordination graph edge-based reward assignment
method. In the non-stationary environment, we modify the MAMAB-based algorithms
proposed in the stationary environment by proposing a practical initialization
method and designing new perturbed terms to adapt to the dynamic environment.
Simulation results are provided to validate the effectiveness of our proposed
algorithms. The effects of different parameters on caching performance are also
discussed.

该研究在未知用户偏好情况下，探究了学习型缓存在小型蜂窝网络（SCN）中的应用。在这个问题中，我们从多智能体多武器老虎机（MAMAB）的角度模拟这个序列决策问题。我们提出几种 MAMAB-Based 算法，在静态和非静态环境下直接学习缓存策略，以实现最小化系统长期传输延迟的目标。该研究验证了所提出的算法的有效性，并进一步讨论了不同参数对缓存性能的影响。