In this work, we evaluate the effectiveness of representation learning
approaches for decision making in visually complex environments. Representation
learning is essential for effective reinforcement learning (RL) from
high-dimensional inputs. Unsupervised representation learning approaches based
on reconstruction, prediction or contrastive learning have shown substantial
learning efficiency gains. Yet, they have mostly been evaluated in clean
laboratory or simulated settings. In contrast, real environments are visually
complex and contain substantial amounts of clutter and distractors.
Unsupervised representations will learn to model such distractors, potentially
impairing the agent's learning efficiency. In contrast, an alternative class of
approaches, which we call task-induced representation learning, leverages task
information such as rewards or demonstrations from prior tasks to focus on
task-relevant parts of the scene and ignore distractors. We investigate the
effectiveness of unsupervised and task-induced representation learning
approaches on four visually complex environments, from Distracting DMControl to
the CARLA driving simulator. For both, RL and imitation learning, we find that
representation learning generally improves sample efficiency on unseen tasks
even in visually complex scenes and that task-induced representations can
double learning efficiency compared to unsupervised alternatives. Code is
available at this https URL.

本研究评估表征学习方法在视觉复杂环境下决策制定中的有效性，并发现任务诱导的表征学习方法可以提高样本效率和学习效率。