A central challenge in reinforcement learning is discovering effective
policies for tasks where rewards are sparsely distributed. We postulate that in
the absence of useful reward signals, an effective exploration strategy should
seek out {\it decision states}. These states lie at critical junctions in the
state space from where the agent can transition to new, potentially unexplored
regions. We propose to learn about decision states from prior experience. By
training a goal-conditioned policy with an information bottleneck, we can
identify decision states by examining where the model actually leverages the
goal state. We find that this simple mechanism effectively identifies decision
states, even in partially observed settings. In effect, the model learns the
sensory cues that correlate with potential subgoals. In new environments, this
model can then identify novel subgoals for further exploration, guiding the
agent through a sequence of potential decision states and through new regions
of the state space.

提出了一种利用信息瓶颈机制学习判决状态的增值方法，通过对观察的状态进行分析，识别有潜力的子目标进行新的探索，并指导代理经过判决状态和状态空间的新区域。