Load balancing and auto scaling are at the core of scalable, contemporary
systems, addressing dynamic resource allocation and service rate adjustments in
response to workload changes. This paper introduces a novel model and
algorithms for tuning load balancers coupled with auto scalers, considering
bursty traffic arriving at finite queues. We begin by presenting the problem as
a weakly coupled Markov Decision Processes (MDP), solvable via a linear program
(LP). However, as the number of control variables of such LP grows
combinatorially, we introduce a more tractable relaxed LP formulation, and
extend it to tackle the problem of online parameter learning and policy
optimization using a two-timescale algorithm based on the LP Lagrangian.

该研究介绍了一种针对具有突发流量的有限队列的负载均衡和自动缩放的新模型和算法，通过线性规划解决了弱耦合的马尔可夫决策过程问题，进一步将其放松为可操作的松弛线性规划，并基于线性规划的拉格朗日函数提出了一个两时间尺度算法来解决在线参数学习和策略优化的问题。

在线学习弱耦合的 MDP 策略用于负载均衡和自动缩放

Online Learning of Weakly Coupled MDP Policies for Load Balancing and  Auto Scaling

Traffic pattern prediction has emerged as a promising approach for
efficiently managing and mitigating the impacts of event-driven bursty traffic
in massive machine-type communication (mMTC) networks. However, achieving
accurate predictions of bursty traffic remains a non-trivial task due to the
inherent randomness of events, and these challenges intensify within live
network environments. Consequently, there is a compelling imperative to design
a lightweight and agile framework capable of assimilating continuously
collected data from the network and accurately forecasting bursty traffic in
mMTC networks. This paper addresses these challenges by presenting a machine
learning-based framework tailored for forecasting bursty traffic in
multi-channel slotted ALOHA networks. The proposed machine learning network
comprises long-term short-term memory (LSTM) and a DenseNet with feed-forward
neural network (FFNN) layers, where the residual connections enhance the
training ability of the machine learning network in capturing complicated
patterns. Furthermore, we develop a new low-complexity online prediction
algorithm that updates the states of the LSTM network by leveraging frequently
collected data from the mMTC network. Simulation results and complexity
analysis demonstrate the superiority of our proposed algorithm in terms of both
accuracy and complexity, making it well-suited for time-critical live
scenarios. We evaluate the performance of the proposed framework in a network
with a single base station and thousands of devices organized into groups with
distinct traffic-generating characteristics. Comprehensive evaluations and
simulations indicate that our proposed machine learning approach achieves a
remarkable $52\%$ higher accuracy in long-term predictions compared to
traditional methods, without imposing additional processing load on the system.

通过使用机器学习网络，本文针对多信道随机接入网络中的突发流量，提出了一种轻量级灵活的框架，并通过在线预测算法实时更新网络状态，实现了对突发流量的准确预测，提高了长期预测精度达到 52%。