Semantic cues and statistical regularities in real-world environment layouts
can improve efficiency for navigation in novel environments. This paper learns
and leverages such semantic cues for navigating to objects of interest in novel
environments, by simply watching YouTube videos. This is challenging because
YouTube videos don't come with labels for actions or goals, and may not even
showcase optimal behavior. Our method tackles these challenges through the use
of Q-learning on pseudo-labeled transition quadruples (image, action, next
image, reward). We show that such off-policy Q-learning from passive data is
able to learn meaningful semantic cues for navigation. These cues, when used in
a hierarchical navigation policy, lead to improved efficiency at the ObjectGoal
task in visually realistic simulations. We observe a relative improvement of
15-83% over end-to-end RL, behavior cloning, and classical methods, while using
minimal direct interaction.

本文通过简单地观看 YouTube 视频，利用 Q-learning 对伪标记过的转移数据进行离线学习来获取语义线索，进而在视觉仿真中实现目标导航任务，相较于端到端 RL、行为克隆和传统方法，使用最少的直接交互能够获得相对提高 15-83% 的效率。