This paper proposes a simple strategy for sim-to-real in Deep-Reinforcement
Learning (DRL) -- called Roll-Drop -- that uses dropout during simulation to
account for observation noise during deployment without explicitly modelling
its distribution for each state. DRL is a promising approach to control robots
for highly dynamic and feedback-based manoeuvres, and accurate simulators are
crucial to providing cheap and abundant data to learn the desired behaviour.
Nevertheless, the simulated data are noiseless and generally show a
distributional shift that challenges the deployment on real machines where
sensor readings are affected by noise. The standard solution is modelling the
latter and injecting it during training; while this requires a thorough system
identification, Roll-Drop enhances the robustness to sensor noise by tuning
only a single parameter. We demonstrate an 80% success rate when up to 25%
noise is injected in the observations, with twice higher robustness than the
baselines. We deploy the controller trained in simulation on a Unitree A1
platform and assess this improved robustness on the physical system.

本文提出一种名为 Roll-Drop 的深度强化学习（DRL）的简单策略，通过在模拟期间使用 dropout 来考虑观察噪声，而不需要为每个状态显式地建模其分布。我们演示当将高达 25％的噪声注入观测后，成功率达到 80％，其鲁棒性比基线提高了一倍。在实际机器上部署受过训练的控制器并评估了所述的改进鲁棒性。