We propose an architecture for learning complex controllable behaviors by
having simple Policies Modulate Trajectory Generators (PMTG), a powerful
combination that can provide both memory and prior knowledge to the controller.
The result is a flexible architecture that is applicable to a class of problems
with periodic motion for which one has an insight into the class of
trajectories that might lead to a desired behavior. We illustrate the basics of
our architecture using a synthetic control problem, then go on to learn
speed-controlled locomotion for a quadrupedal robot by using Deep Reinforcement
Learning and Evolutionary Strategies. We demonstrate that a simple linear
policy, when paired with a parametric Trajectory Generator for quadrupedal
gaits, can induce walking behaviors with controllable speed from 4-dimensional
IMU observations alone, and can be learned in under 1000 rollouts. We also
transfer these policies to a real robot and show locomotion with controllable
forward velocity.

本文提出了一种利用简单策略调制轨迹生成器的体系结构，通过深度强化学习和进化策略学习出具有可控行为的四足机器人运动，实现了从 IMU 观测数据中控制速度的步态诱导。