Reinforcement learning (RL) is a powerful tool for optimal control that has
found great success in Atari games, the game of Go, robotic control, and
building optimization. RL is also very brittle; agents often overfit to their
training environment and fail to generalize to new settings. Unsupervised
environment design (UED) has been proposed as a solution to this problem, in
which the agent trains in environments that have been specially selected to
help it learn. Previous UED algorithms focus on trying to train an RL agent
that generalizes across a large distribution of environments. This is not
necessarily desirable when we wish to prioritize performance in one environment
over others. In this work, we will be examining the setting of robust RL
building control, where we wish to train an RL agent that prioritizes
performing well in normal weather while still being robust to extreme weather
conditions. We demonstrate a novel UED algorithm, ActivePLR, that uses
uncertainty-aware neural network architectures to generate new training
environments at the limit of the RL agent's ability while being able to
prioritize performance in a desired base environment. We show that ActivePLR is
able to outperform state-of-the-art UED algorithms in minimizing energy usage
while maximizing occupant comfort in the setting of building control.

在建筑控制领域，本研究提出了一种新颖的 UED 算法 ActivePLR，通过使用具有不确定性感知的神经网络结构，在 RL 代理的能力极限处生成新的训练环境，并能够优先考虑在基准环境中的性能，表明 ActivePLR 能够在最小化能源消耗的同时最大化居住者舒适度方面优于最先进的 UED 算法。