Predictive autoscaling (autoscaling with workload forecasting) is an important mechanism that supports autonomous adjustment of computing resources in accordance with fluctuating workload demands in the Cloud. In recent works, Reinforcement Learning (RL) has been introduced as a promising approach to learn the resource management policies to guide the scaling actions under the dynamic and uncertain cloud environment. However, RL methods face the following challenges in steering predictive autoscaling, such as lack of accuracy in decision-making, inefficient sampling and significant variability in workload patterns that may cause policies to fail at test time. To this end, we propose an end-to-end predictive meta model-based RL algorithm, aiming to optimally allocate resource to maintain a stable CPU utilization level, which incorporates a specially-designed deep periodic workload prediction model as the input and embeds the Neural Process to guide the learning of the optimal scaling actions over numerous application services in the Cloud. Our algorithm not only ensures the predictability and accuracy of the scaling strategy, but also enables the scaling decisions to adapt to the changing workloads with high sample efficiency. Our method has achieved significant performance improvement compared to the existing algorithms and has been deployed online at Alipay, supporting the autoscaling of applications for the world-leading payment platform.

本研究提出了一种基于端对端预测元模型的强化学习算法，通过神经过程的指导，将计算资源优化分配到应用程序服务中，以维持稳定的 CPU 利用水平，以应对动态变化的工作负载。该算法不仅确保了扩展策略的可预测性和准确性，还能够在高样本效率下自适应地适应不断变化的工作负载，经过阿里平台的在线部署，为世界领先的支付平台提供了应用程序的自动化扩缩容支持，达到了显著的性能提升。

云中基于元强化学习方法的预测自动伸缩