Exploring in environments with high-dimensional observations is hard. One
promising approach for exploration is to use intrinsic rewards, which often
boils down to estimating "novelty" of states, transitions, or trajectories with
deep networks. Prior works have shown that conditional prediction objectives
such as masked autoencoding can be seen as stochastic estimation of
pseudo-likelihood. We show how this perspective naturally leads to a unified
view on existing intrinsic reward approaches: they are special cases of
conditional prediction, where the estimation of novelty can be seen as
pseudo-likelihood estimation with different mask distributions. From this view,
we propose a general framework for deriving intrinsic rewards -- Masked Input
Modeling for Exploration (MIMEx) -- where the mask distribution can be flexibly
tuned to control the difficulty of the underlying conditional prediction task.
We demonstrate that MIMEx can achieve superior results when compared against
competitive baselines on a suite of challenging sparse-reward visuomotor tasks.

提出了一种用于探索的通用框架 Masked Input Modeling for Exploration (MIMEx)，它能够通过灵活调整掩码分布来控制条件预测任务的难度，并在一系列挑战性的稀疏奖励视觉运动任务中取得了优异的结果。