We present a hierarchical framework that combines model-based control and
reinforcement learning (RL) to synthesize robust controllers for a quadruped
(the Unitree Laikago). The system consists of a high-level controller that
learns to choose from a set of primitives in response to changes in the
environment and a low-level controller that utilizes an established control
method to robustly execute the primitives. Our framework learns a controller
that can adapt to challenging environmental changes on the fly, including novel
scenarios not seen during training. The learned controller is up to 85~percent
more energy efficient and is more robust compared to baseline methods. We also
deploy the controller on a physical robot without any randomization or
adaptation scheme.

本研究提出了一个层次框架，将基于模型的控制和强化学习相结合，为四足动物（Unitree Laikago）合成鲁棒控制器。通过学习选择一套原语响应环境中的变化，使其适应复杂的环境变化，并具有更高的能源效率和更强的鲁棒性。

为实现稳健高效的腿式运动而学习接触自适应控制器

Learning a Contact-Adaptive Controller for Robust, Efficient Legged  Locomotion

Designing agile locomotion for quadruped robots often requires extensive
expertise and tedious manual tuning. In this paper, we present a system to
automate this process by leveraging deep reinforcement learning techniques. Our
system can learn quadruped locomotion from scratch using simple reward signals.
In addition, users can provide an open loop reference to guide the learning
process when more control over the learned gait is needed. The control policies
are learned in a physics simulator and then deployed on real robots. In
robotics, policies trained in simulation often do not transfer to the real
world. We narrow this reality gap by improving the physics simulator and
learning robust policies. We improve the simulation using system
identification, developing an accurate actuator model and simulating latency.
We learn robust controllers by randomizing the physical environments, adding
perturbations and designing a compact observation space. We evaluate our system
on two agile locomotion gaits: trotting and galloping. After learning in
simulation, a quadruped robot can successfully perform both gaits in the real
world.

本文提出了一种利用深度强化学习技术自动化四足机器人运动设计过程的系统，能够从简单的奖励信号中学习四足运动，并可提供开环参考进行学习过程的控制，采用系统辨识来改进物理模拟器，利用物理环境随机与扰动设计控制器，并在物理模拟器中进行评估，成功在现实世界中部署。