Client-side video players employ adaptive bitrate (ABR) algorithms to
optimize user quality of experience (QoE). We evaluate recently proposed
RL-based ABR methods in Facebook's web-based video streaming platform.
Real-world ABR contains several challenges that requires customized designs
beyond off-the-shelf RL algorithms -- we implement a scalable neural network
architecture that supports videos with arbitrary bitrate encodings; we design a
training method to cope with the variance resulting from the stochasticity in
network conditions; and we leverage constrained Bayesian optimization for
reward shaping in order to optimize the conflicting QoE objectives. In a
week-long worldwide deployment with more than 30 million video streaming
sessions, our RL approach outperforms the existing human-engineered ABR
algorithms.

在 Facebook 视频流平台，我们评估最新提出的基于 RL 的自适应比特率算法，采用可扩展的神经网络架构设计，利用贝叶斯优化进行奖励塑形，优化用户体验目标，以应对网络环境的随机性方差。在全球范围内超过 3000 万次视频流会话中，我们的 RL 方法优于现有的人工设计的算法。