In this letter, we present a method for integrating central pattern
generators (CPGs), i.e. systems of coupled oscillators, into the deep
reinforcement learning (DRL) framework to produce robust and omnidirectional
quadruped locomotion. The agent learns to directly modulate the intrinsic
oscillator setpoints (amplitude and frequency) and coordinate rhythmic behavior
among different oscillators. This approach also allows the use of DRL to
explore questions related to neuroscience, namely the role of descending
pathways, interoscillator couplings, and sensory feedback in gait generation.
We train our policies in simulation and perform a sim-to-real transfer to the
Unitree A1 quadruped, where we observe robust behavior to disturbances unseen
during training, most notably to a dynamically added 13.75 kg load representing
115% of the nominal quadruped mass. We test several different observation
spaces based on proprioceptive sensing and show that our framework is
deployable with no domain randomization and very little feedback, where along
with the oscillator states, it is possible to provide only contact booleans in
the observation space. Video results can be found at
this https URL.

本文提出了一种方法，将中心模式发生器（CPG），即振荡器系统，整合到深度强化学习（DRL）框架中，以产生稳健且全向的四足动物运动。模型直接调制内在振荡器设定点的振幅和频率，以及协调不同振荡器间的节律性行为，并可以使用 DRL 探索神经科学相关问题的答案，例如下行途径、振荡器间的耦合和步态生成中的感觉反馈。我们在模拟运用中训练了政策，然后进行 sim-to-real 转移至 Unitree A1，其中我们发现模型行为的鲁棒性优于训练时未见过的干扰，最重要的是动态添加的 13.75 公斤的负载，代表着 115％的名义四足动物质量。我们测试了几种基于本体感官的观察空间，并展示了我们的框架可以在没有任何领域随机化和非常少的反馈的情况下进行部署，它可以提供振荡器状态和触点布尔值。视频结果可在此 https URL 找到。