Representation learning is a central challenge across a range of machine
learning areas. In reinforcement learning, effective and functional
representations have the potential to tremendously accelerate learning progress
and solve more challenging problems. Most prior work on representation learning
has focused on generative approaches, learning representations that capture all
underlying factors of variation in the observation space in a more disentangled
or well-ordered manner. In this paper, we instead aim to learn functionally
salient representations: representations that are not necessarily complete in
terms of capturing all factors of variation in the observation space, but
rather aim to capture those factors of variation that are important for
decision making -- that are "actionable." These representations are aware of
the dynamics of the environment, and capture only the elements of the
observation that are necessary for decision making rather than all factors of
variation, without explicit reconstruction of the observation. We show how
these representations can be useful to improve exploration for sparse reward
problems, to enable long horizon hierarchical reinforcement learning, and as a
state representation for learning policies for downstream tasks. We evaluate
our method on a number of simulated environments, and compare it to prior
methods for representation learning, exploration, and hierarchical
reinforcement learning.

本文研究功能性显著表征的强化学习方法，可以用于改善稀疏奖励问题的探索、实现具有长期视野的分层强化学习和作为下游任务的学习策略的状态表征。通过在多个虚拟环境中对比实验，表明该方法在表征学习、探索和分层强化学习方面具有优势。