The use of learning-based methods for optimizing cellular radio access
networks (RAN) has received increasing attention in recent years. This
coincides with a rapid increase in the number of cell sites worldwide, driven
largely by dramatic growth in cellular network traffic. Training and
maintaining learned models that work well across a large number of cell sites
has thus become a pertinent problem. This paper proposes a scalable framework
for constructing a reinforcement learning policy bank that can perform RAN
optimization across a large number of cell sites with varying traffic patterns.
Central to our framework is a novel application of anomaly detection techniques
to assess the compatibility between sites (tasks) and the policy bank. This
allows our framework to intelligently identify when a policy can be reused for
a task, and when a new policy needs to be trained and added to the policy bank.
Our results show that our approach to compatibility assessment leads to an
efficient use of computational resources, by allowing us to construct a
performant policy bank without exhaustively training on all tasks, which makes
it applicable under real-world constraints.

使用基于学习的方法优化移动通信网络 (RAN) 在最近几年得到了越来越多的关注，本文提出了一个可扩展的框架，用于构建一个策略库，可以在大量具有不同流量模式的基站上进行 RAN 优化，并将异常检测技术应用于判断任务与策略库的兼容性，以高效地使用计算资源。

基于强化学习的 RAN 优化中的可扩展任务分组的异常检测

Anomaly Detection for Scalable Task Grouping in Reinforcement  Learning-based RAN Optimization

With the continuous growth in communication network complexity and traffic
volume, communication load balancing solutions are receiving increasing
attention. Specifically, reinforcement learning (RL)-based methods have shown
impressive performance compared with traditional rule-based methods. However,
standard RL methods generally require an enormous amount of data to train, and
generalize poorly to scenarios that are not encountered during training. We
propose a policy reuse framework in which a policy selector chooses the most
suitable pre-trained RL policy to execute based on the current traffic
condition. Our method hinges on a policy bank composed of policies trained on a
diverse set of traffic scenarios. When deploying to an unknown traffic
scenario, we select a policy from the policy bank based on the similarity
between the previous-day traffic of the current scenario and the traffic
observed during training. Experiments demonstrate that this framework can
outperform classical and adaptive rule-based methods by a large margin.

本研究提出了一种基于强化学习的策略重用框架，通过在各种交通场景下训练和存储策略，并结合流量条件，选择最适合的预训练策略以更好地解决通信网络负载均衡问题。实验结果表明，这种方法比传统的基于规则和适应性方法表现更出色。