In this work we explore a new approach for robots to teach themselves about the world simply by observing it. In particular we investigate the effectiveness of learning task-agnostic representations for continuous control tasks. We extend Time-Contrastive Networks (TCN) that learn from visual observations by embedding multiple frames jointly in the embedding space as opposed to a single frame. We show that by doing so, we are now able to encode both position and velocity attributes significantly more accurately. We test the usefulness of this self-supervised approach in a reinforcement learning setting. We show that the representations learned by agents observing themselves take random actions, or other agents perform tasks successfully, can enable the learning of continuous control policies using algorithms like Proximal Policy Optimization (PPO) using only the learned embeddings as input. We also demonstrate significant improvements on the real-world Pouring dataset with a relative error reduction of 39.4% for motion attributes and 11.1% for static attributes compared to the single-frame baseline. Video results are available at https://sites.google.com/view/actionablerepresentations .

本文研究了使用自我监督学习来学习连续控制任务的对抗学习方法。通过将多个帧联合嵌入到嵌入空间中，我们扩展了时间对比网络（TCN），从而能够更准确地编码位置和速度属性。我们证明了这种方法在强化学习任务中有效，可以通过仅使用学习到的嵌入作为输入，使用像Proximal Policy Optimization（PPO）这样的算法来学习连续控制策略。

从视觉观察中学习可操作的表征