Demand response (DR) for residential and small commercial buildings is
estimated to account for as much as 65% of the total energy savings potential
of DR, and previous work shows that a fully automated Energy Management System
(EMS) is a necessary prerequisite to DR in these areas. In this paper, we
propose a novel EMS formulation for DR problems in these sectors. Specifically,
we formulate a fully automated EMS's rescheduling problem as a reinforcement
learning (RL) problem, and argue that this RL problem can be approximately
solved by decomposing it over device clusters. Compared with existing
formulations, our new formulation (1) does not require explicitly modeling the
user's dissatisfaction on job rescheduling, (2) enables the EMS to
self-initiate jobs, (3) allows the user to initiate more flexible requests and
(4) has a computational complexity linear in the number of devices. We also
demonstrate the simulation results of applying Q-learning, one of the most
popular and classical RL algorithms, to a representative example.

本文提出了一种新型的供需响应问题的全自动能源管理系统的调度问题的强化学习方法，并以 Q 学习算法为例，通过对代表性例子的模拟结果进行了演示。