As Exascale computing becomes a reality, the energy needs of compute nodes in cloud data centers will continue to grow. A common approach to reducing this energy demand is to limit the power consumption of hardware components when workloads are experiencing bottlenecks elsewhere in the system. However, designing a resource controller capable of detecting and limiting power consumption on-the-fly is a complex issue and can also adversely impact application performance. In this paper, we explore the use of Reinforcement Learning (RL) to design a power capping policy on cloud compute nodes using observations on current power consumption and instantaneous application performance (heartbeats). By leveraging the Argo Node Resource Management (NRM) software stack in conjunction with the Intel Running Average Power Limit (RAPL) hardware control mechanism, we design an agent to control the maximum supplied power to processors without compromising on application performance. Employing a Proximal Policy Optimization (PPO) agent to learn an optimal policy on a mathematical model of the compute nodes, we demonstrate and evaluate using the STREAM benchmark how a trained agent running on actual hardware can take actions by balancing power consumption and application performance.

当Exascale计算成为现实时，云数据中心的计算节点能源需求将继续增长。在系统其他方面经历瓶颈时，降低硬件组件的功耗是减少能源需求的常见方法之一。然而，设计一个能够实时检测和限制功耗的资源控制器是一个复杂的问题，可能会对应用性能产生不利影响。本文中，我们探索使用强化学习（RL）设计云计算节点上的功耗限制策略，利用当前功耗和瞬时应用性能（心跳）的观测。通过将Argo Node Resource Management（NRM）软件堆栈与Intel Running Average Power Limit（RAPL）硬件控制机制相结合，设计一个代理程序来控制处理器的最大供电功率，而不会影响应用性能。利用Proximal Policy Optimization（PPO）代理程序在计算节点的数学模型上学习最优策略，我们使用STREAM基准测试演示和评估了在实际硬件上运行的经过训练的代理程序如何通过平衡功耗和应用性能来采取行动。

数据中心计算节点功耗降低的性能感知增强学习方法