Media streaming is the dominant application over wireless edge (access)
networks. The increasing softwarization of such networks has led to efforts at
intelligent control, wherein application-specific actions may be dynamically
taken to enhance the user experience. The goal of this work is to develop and
demonstrate learning-based policies for optimal decision making to determine
which clients to dynamically prioritize in a video streaming setting. We
formulate the policy design question as a constrained Markov decision problem
(CMDP), and observe that by using a Lagrangian relaxation we can decompose it
into single-client problems. Further, the optimal policy takes a threshold form
in the video buffer length, which enables us to design an efficient constrained
reinforcement learning (CRL) algorithm to learn it. Specifically, we show that
a natural policy gradient (NPG) based algorithm that is derived using the
structure of our problem converges to the globally optimal policy. We then
develop a simulation environment for training, and a real-world intelligent
controller attached to a WiFi access point for evaluation. We empirically show
that the structured learning approach enables fast learning. Furthermore, such
a structured policy can be easily deployed due to low computational complexity,
leading to policy execution taking only about 15$\mu$s. Using YouTube streaming
experiments in a resource constrained scenario, we demonstrate that the CRL
approach can increase QoE by over 30%.

通过使用学习型策略来确定在视频流媒体环境中哪些客户端应该动态优先考虑，以提升用户体验和增加 30% 的 QoE，并使用低计算复杂度的结构化策略进行快速学习。