Media streaming is the dominant application over wireless edge (access)
networks. The increasing softwarization of such networks has led to efforts at
intelligent control, wherein application-specific actions may be dynamically
taken to enhance the user experience. The goal of this work is to develop and
demonstrate learning-based policies for optimal decision making to determine
which clients to dynamically prioritize in a video streaming setting. We
formulate the policy design question as a constrained Markov decision problem
(CMDP), and observe that by using a Lagrangian relaxation we can decompose it
into single-client problems. Further, the optimal policy takes a threshold form
in the video buffer length, which enables us to design an efficient constrained
reinforcement learning (CRL) algorithm to learn it. Specifically, we show that
a natural policy gradient (NPG) based algorithm that is derived using the
structure of our problem converges to the globally optimal policy. We then
develop a simulation environment for training, and a real-world intelligent
controller attached to a WiFi access point for evaluation. We empirically show
that the structured learning approach enables fast learning. Furthermore, such
a structured policy can be easily deployed due to low computational complexity,
leading to policy execution taking only about 15$\mu$s. Using YouTube streaming
experiments in a resource constrained scenario, we demonstrate that the CRL
approach can increase QoE by over 30%.

通过使用学习型策略来确定在视频流媒体环境中哪些客户端应该动态优先考虑，以提升用户体验和增加 30% 的 QoE，并使用低计算复杂度的结构化策略进行快速学习。

无线边缘多媒体流媒体结构化强化学习

Structured Reinforcement Learning for Media Streaming at the Wireless  Edge

Learning provides a powerful tool for vision-based navigation, but the
capabilities of learning-based policies are constrained by limited training
data. If we could combine data from all available sources, including multiple
kinds of robots, we could train more powerful navigation models. In this paper,
we study how a general goal-conditioned model for vision-based navigation can
be trained on data obtained from many distinct but structurally similar robots,
and enable broad generalization across environments and embodiments. We analyze
the necessary design decisions for effective data sharing across robots,
including the use of temporal context and standardized action spaces, and
demonstrate that an omnipolicy trained from heterogeneous datasets outperforms
policies trained on any single dataset. We curate 60 hours of navigation
trajectories from 6 distinct robots, and deploy the trained GNM on a range of
new robots, including an underactuated quadrotor. We find that training on
diverse data leads to robustness against degradation in sensing and actuation.
Using a pre-trained navigation model with broad generalization capabilities can
bootstrap applications on novel robots going forward, and we hope that the GNM
represents a step in that direction. For more information on the datasets,
code, and videos, please check out
this http URL.

本文研究了如何通过结构相似的多种机器人的数据来训练基于视觉的导航通用目标条件模型，并分析了有效数据共享的必要设计决策，证明了从异构数据集训练的普适策略优于任何单一数据集训练的策略。