End-to-end delay is a critical attribute of quality of service (QoS) in
application domains such as cloud computing and computer networks. This metric
is particularly important in tandem service systems, where the end-to-end
service is provided through a chain of services. Service-rate control is a
common mechanism for providing QoS guarantees in service systems. In this
paper, we introduce a reinforcement learning-based (RL-based) service-rate
controller that provides probabilistic upper-bounds on the end-to-end delay of
the system, while preventing the overuse of service resources. In order to have
a general framework, we use queueing theory to model the service systems.
However, we adopt an RL-based approach to avoid the limitations of
queueing-theoretic methods. In particular, we use Deep Deterministic Policy
Gradient (DDPG) to learn the service rates (action) as a function of the queue
lengths (state) in tandem service systems. In contrast to existing RL-based
methods that quantify their performance by the achieved overall reward, which
could be hard to interpret or even misleading, our proposed controller provides
explicit probabilistic guarantees on the end-to-end delay of the system. The
evaluations are presented for a tandem queueing system with non-exponential
inter-arrival and service times, the results of which validate our controller's
capability in meeting QoS constraints.

本文介绍了一种基于强化学习的服务速率控制器，该控制器通过使用深度确定性策略梯度 (DDPG) 算法作为函数，将服务速率 (行动) 学习为串联服务系统中队列长度 (状态) 的函数，并提供关于系统端到端延迟的概率性保证。