This paper investigates learning-based caching in small-cell networks (SCNs)
when user preference is unknown. The goal is to optimize the cache placement in
each small base station (SBS) for minimizing the system long-term transmission
delay. We model this sequential multi-agent decision making problem in a
multi-agent multi-armed bandit (MAMAB) perspective. Rather than estimating user
preference first and then optimizing the cache strategy, we propose several
MAMAB-based algorithms to directly learn the cache strategy online in both
stationary and non-stationary environment. In the stationary environment, we
first propose two high-complexity agent-based collaborative MAMAB algorithms
with performance guarantee. Then we propose a low-complexity distributed MAMAB
which ignores the SBS coordination. To achieve a better balance between SBS
coordination gain and computational complexity, we develop an edge-based
collaborative MAMAB with the coordination graph edge-based reward assignment
method. In the non-stationary environment, we modify the MAMAB-based algorithms
proposed in the stationary environment by proposing a practical initialization
method and designing new perturbed terms to adapt to the dynamic environment.
Simulation results are provided to validate the effectiveness of our proposed
algorithms. The effects of different parameters on caching performance are also
discussed.

该研究在未知用户偏好情况下，探究了学习型缓存在小型蜂窝网络（SCN）中的应用。在这个问题中，我们从多智能体多武器老虎机（MAMAB）的角度模拟这个序列决策问题。我们提出几种 MAMAB-Based 算法，在静态和非静态环境下直接学习缓存策略，以实现最小化系统长期传输延迟的目标。该研究验证了所提出的算法的有效性，并进一步讨论了不同参数对缓存性能的影响。

小区缓存的协作多智体多臂赌博学习

Collaborative Multi-Agent Multi-Armed Bandit Learning for Small-Cell  Caching

We consider the problem of storing segments of encoded versions of content
files in a set of base stations located in a communication cell. These base
stations work in conjunction with the main base station of the cell. Users move
randomly across the space based on a discrete-time Markov chain model. At each
time slot each user accesses a single base station based on it's current
position and it can download only a part of the content stored in it, depending
on the time slot duration. We assume that file requests must be satisfied
within a given time deadline in order to be successful. If the amount of the
downloaded (encoded) data by the accessed base stations when the time deadline
expires does not suffice to recover the requested file, the main base station
of the cell serves the request. Our aim is to find the storage allocation that
minimizes the probability of using the main base station for file delivery.
This problem is intractable in general. However, we show that the optimal
solution of the problem can be efficiently attained in case that the time
deadline is small. To tackle the general case, we propose a distributed
approximation algorithm based on large deviation inequalities. Systematic
experiments on a real world data set demonstrate the effectiveness of our
proposed algorithms. Index Terms: Mobility-aware Caching, Markov Chain, MDS
Coding, Small-cell Networks

该研究主要研究如何在具有移动性的小型网络中存储内容文件的编码版本，并提出了分布式逼近算法来解决该问题。在实验中，使用基于大偏差不等式的方法使得研究结论在实际应用中更加精准。