We consider the problem of demand-side energy management, where each
household is equipped with a smart meter that is able to schedule home
appliances online. The goal is to minimize the overall cost under a real-time
pricing scheme. While previous works have introduced centralized approaches in
which the scheduling algorithm has full observability, we propose the
formulation of a smart grid environment as a Markov game. Each household is a
decentralized agent with partial observability, which allows scalability and
privacy-preservation in a realistic setting. The grid operator produces a price
signal that varies with the energy demand. We propose an extension to a
multi-agent, deep actor-critic algorithm to address partial observability and
the perceived non-stationarity of the environment from the agent's viewpoint.
This algorithm learns a centralized critic that coordinates training of
decentralized agents. Our approach thus uses centralized learning but
decentralized execution. Simulation results show that our online deep
reinforcement learning method can reduce both the peak-to-average ratio of
total energy consumed and the cost of electricity for all households based
purely on instantaneous observations and a price signal.

本文针对需求侧能源管理问题，提出了一种基于智能电网环境的多智能体深度强化学习算法，可以在保证可扩展性和隐私保护的前提下利用即时观测和价格信号来最小化总体成本，并降低家庭总能耗的峰谷比和电力费用。

基于多智能体深度 Actor-Critic 学习的智能电网需求侧调度

Demand-Side Scheduling Based on Multi-Agent Deep Actor-Critic Learning  for Smart Grids

Motivated by the application of real-time pricing in e-commerce platforms, we
consider the problem of revenue-maximization in a setting where the seller can
leverage contextual information describing the customer's history and the
product's type to predict her valuation of the product. However, her true
valuation is unobservable to the seller, only binary outcome in the form of
success-failure of a transaction is observed. Unlike in usual contextual bandit
settings, the optimal price/arm given a covariate in our setting is sensitive
to the detailed characteristics of the residual uncertainty distribution. We
develop a semi-parametric model in which the residual distribution is
non-parametric and provide the first algorithm which learns both regression
parameters and residual distribution with $\tilde O(\sqrt{n})$ regret. We
empirically test a scalable implementation of our algorithm and observe good
performance.

基于实时定价与上下文信息，本文提出了一种半参数模型，能够通过回归参数和残差分布来最大化收益，考虑到了细节特征。