With the continuous growth in communication network complexity and traffic
volume, communication load balancing solutions are receiving increasing
attention. Specifically, reinforcement learning (RL)-based methods have shown
impressive performance compared with traditional rule-based methods. However,
standard RL methods generally require an enormous amount of data to train, and
generalize poorly to scenarios that are not encountered during training. We
propose a policy reuse framework in which a policy selector chooses the most
suitable pre-trained RL policy to execute based on the current traffic
condition. Our method hinges on a policy bank composed of policies trained on a
diverse set of traffic scenarios. When deploying to an unknown traffic
scenario, we select a policy from the policy bank based on the similarity
between the previous-day traffic of the current scenario and the traffic
observed during training. Experiments demonstrate that this framework can
outperform classical and adaptive rule-based methods by a large margin.

本研究提出了一种基于强化学习的策略重用框架，通过在各种交通场景下训练和存储策略，并结合流量条件，选择最适合的预训练策略以更好地解决通信网络负载均衡问题。实验结果表明，这种方法比传统的基于规则和适应性方法表现更出色。

未见流量场景下通信负载平衡的策略重用

Policy Reuse for Communication Load Balancing in Unseen Traffic Scenarios

Communication load balancing aims to balance the load between different
available resources, and thus improve the quality of service for network
systems. After formulating the load balancing (LB) as a Markov decision process
problem, reinforcement learning (RL) has recently proven effective in
addressing the LB problem. To leverage the benefits of classical RL for load
balancing, however, we need an explicit reward definition. Engineering this
reward function is challenging, because it involves the need for expert
knowledge and there lacks a general consensus on the form of an optimal reward
function. In this work, we tackle the communication load balancing problem from
an inverse reinforcement learning (IRL) approach. To the best of our knowledge,
this is the first time IRL has been successfully applied in the field of
communication load balancing. Specifically, first, we infer a reward function
from a set of demonstrations, and then learn a reinforcement learning load
balancing policy with the inferred reward function. Compared to classical
RL-based solution, the proposed solution can be more general and more suitable
for real-world scenarios. Experimental evaluations implemented on different
simulated traffic scenarios have shown our method to be effective and better
than other baselines by a considerable margin.

本文提出了一种采用逆向强化学习方法来解决通信负载平衡问题的方案，并通过从一组演示中推断出奖励函数和学习负载平衡策略的方法，取得了在真实场景中更适用和更具一般性的效果。实验表明所提出的方案优于其他基线方法。