Varying dynamics parameters in simulation is a popular Domain Randomization
(DR) approach for overcoming the reality gap in Reinforcement Learning (RL).
Nevertheless, DR heavily hinges on the choice of the sampling distribution of
the dynamics parameters, since high variability is crucial to regularize the
agent's behavior but notoriously leads to overly conservative policies when
randomizing excessively. In this paper, we propose a novel approach to address
sim-to-real transfer, which automatically shapes dynamics distributions during
training in simulation without requiring real-world data. We introduce DOmain
RAndomization via Entropy MaximizatiON (DORAEMON), a constrained optimization
problem that directly maximizes the entropy of the training distribution while
retaining generalization capabilities. In achieving this, DORAEMON gradually
increases the diversity of sampled dynamics parameters as long as the
probability of success of the current policy is sufficiently high. We
empirically validate the consistent benefits of DORAEMON in obtaining highly
adaptive and generalizable policies, i.e. solving the task at hand across the
widest range of dynamics parameters, as opposed to representative baselines
from the DR literature. Notably, we also demonstrate the Sim2Real applicability
of DORAEMON through its successful zero-shot transfer in a robotic manipulation
setup under unknown real-world parameters.

通过在模拟中自动调节动力学分布而无需真实世界数据，我们提出了 DOmain RAndomization via Entropy MaximizatiON (DORAEMON) 方法，该方法通过增加采样动力学参数的多样性来提高自适应性和泛化能力，并在未知真实参数下成功进行零 - shot 迁移最大化目标任务的解决。