Sparse rewards are double-edged training signals in reinforcement learning:
easy to design but hard to optimize. Intrinsic motivation guidances have thus
been developed toward alleviating the resulting exploration problem. They
usually incentivize agents to look for new states through novelty signals. Yet,
such methods encourage exhaustive exploration of the state space rather than
focusing on the environment's salient interaction opportunities. We propose a
new exploration method, called Don't Do What Doesn't Matter (DoWhaM), shifting
the emphasis from state novelty to state with relevant actions. While most
actions consistently change the state when used, \textit{e.g.} moving the
agent, some actions are only effective in specific states, \textit{e.g.},
\emph{opening} a door, \emph{grabbing} an object. DoWhaM detects and rewards
actions that seldom affect the environment. We evaluate DoWhaM on the
procedurally-generated environment MiniGrid, against state-of-the-art methods
and show that DoWhaM greatly reduces sample complexity.

本文提出了一种名为 DoWhaM 的新探索方法，将重点从状态新颖性转移到具有相关动作的状态，以减少样本复杂度并解决强化学习中的探索问题。