Deep reinforcement learning (RL) based controllers for legged robots have demonstrated impressive robustness for walking in different environments for several robot platforms. To enable the application of RL policies for humanoid robots in real-world settings, it is crucial to build a system that can achieve robust walking in any direction, on 2D and 3D terrains, and be controllable by a user-command. In this paper, we tackle this problem by learning a policy to follow a given step sequence. The policy is trained with the help of a set of procedurally generated step sequences (also called footstep plans). We show that simply feeding the upcoming 2 steps to the policy is sufficient to achieve omnidirectional walking, turning in place, standing, and climbing stairs. Our method employs curriculum learning on the complexity of terrains, and circumvents the need for reference motions or pre-trained weights. We demonstrate the application of our proposed method to learn RL policies for 2 new robot platforms - HRP5P and JVRC-1 - in the MuJoCo simulation environment. The code for training and evaluation is available online.

本研究基于深度强化学习控制器的支持下，通过学习遵循给定步长序列的原则，利用程序生成的步长方案实现了全向行走、原地转弯、站立和爬楼梯等多项功能，同时具备适应性强、无需预训练权重、不依赖参考动作等特点，为改进人形机器人在现实环境中行走鲁棒性提供了新思路。

针对人形机器人的计划脚步双足行走学习