Generalization has been a long-standing challenge for reinforcement learning
(RL). Visual RL, in particular, can be easily distracted by irrelevant factors
in high-dimensional observation space. In this work, we consider robust policy
learning which targets zero-shot generalization to unseen visual environments
with large distributional shift. We propose SECANT, a novel self-expert cloning
technique that leverages image augmentation in two stages to decouple robust
representation learning from policy optimization. Specifically, an expert
policy is first trained by RL from scratch with weak augmentations. A student
network then learns to mimic the expert policy by supervised learning with
strong augmentations, making its representation more robust against visual
variations compared to the expert. Extensive experiments demonstrate that
SECANT significantly advances the state of the art in zero-shot generalization
across 4 challenging domains. Our average reward improvements over prior SOTAs
are: DeepMind Control (+26.5%), robotic manipulation (+337.8%), vision-based
autonomous driving (+47.7%), and indoor object navigation (+15.8%). Code
release and video are available at this https URL

本论文提出了一种名为 SECANT 的新型自我专家复制技术，旨在学习从弱增强到强增强的映像数据，以提高其对视觉变化的鲁棒性，在包括 DeepMind Control，机器人操作，基于视觉的自主驾驶和室内物体导航在内的四个具有挑战性的领域，SECANT 显着提高了 SOTA 的平均奖励。