While most recent advancements in legged robot control have been driven by
model-free reinforcement learning, we explore the potential of differentiable
simulation. Differentiable simulation promises faster convergence and more
stable training by computing low-variant first-order gradients using the robot
model, but so far, its use for legged robot control has remained limited to
simulation. The main challenge with differentiable simulation lies in the
complex optimization landscape of robotic tasks due to discontinuities in
contact-rich environments, e.g., quadruped locomotion. This work proposes a
new, differentiable simulation framework to overcome these challenges. The key
idea involves decoupling the complex whole-body simulation, which may exhibit
discontinuities due to contact, into two separate continuous domains.
Subsequently, we align the robot state resulting from the simplified model with
a more precise, non-differentiable simulator to maintain sufficient simulation
accuracy. Our framework enables learning quadruped walking in minutes using a
single simulated robot without any parallelization. When augmented with GPU
parallelization, our approach allows the quadruped robot to master diverse
locomotion skills, including trot, pace, bound, and gallop, on challenging
terrains in minutes. Additionally, our policy achieves robust locomotion
performance in the real world zero-shot. To the best of our knowledge, this
work represents the first demonstration of using differentiable simulation for
controlling a real quadruped robot. This work provides several important
insights into using differentiable simulations for legged locomotion in the
real world.

提出了可用于实际四足机器人控制的可微分仿真框架，能够在短时间内实现四足机器人的多种暴走技能，以及在现实世界中的鲁棒步态表现。