Learning dexterous locomotion policy for legged robots is becoming
increasingly popular due to its ability to handle diverse terrains and resemble
intelligent behaviors. However, joint manipulation of moving objects and
locomotion with legs, such as playing soccer, receive scant attention in the
learning community, although it is natural for humans and smart animals. A key
challenge to solve this multitask problem is to infer the objectives of
locomotion from the states and targets of the manipulated objects. The implicit
relation between the object states and robot locomotion can be hard to capture
directly from the training experience. We propose adding a feedback control
block to compute the necessary body-level movement accurately and using the
outputs as dynamic joint-level locomotion supervision explicitly. We further
utilize an improved ball dynamic model, an extended context-aided estimator,
and a comprehensive ball observer to facilitate transferring policy learned in
simulation to the real world. We observe that our learning scheme can not only
make the policy network converge faster but also enable soccer robots to
perform sophisticated maneuvers like sharp cuts and turns on flat surfaces, a
capability that was lacking in previous methods. Video and code are available
at this https URL

为了处理不同的地形和模仿智能行为，学习四肢机器人熟练行走策略变得越来越受欢迎。尽管对于人类和智能动物来说这是一种自然行为，但在学习社区中，对于操纵移动物体和通过腿部进行运动（如踢足球）的联合操作却受到了少量关注。解决这个多任务问题的关键挑战是根据操纵对象的状态和目标来推断行走的目标，这两者之间的隐含关系很难直接从训练经验中捕捉。我们提议添加一个反馈控制模块，以准确计算所需的身体级运动，并将输出作为关节级行走监督显式地使用。我们进一步利用改进的球体动态模型、扩展的上下文辅助估计器和综合的球体观察器来促进将在仿真中学习的策略转化到现实世界中。我们观察到我们的学习方案不仅可以使策略网络更快地收敛，还可以使足球机器人在平坦表面上执行精巧的运动，如急转弯和转向，这是以前方法所缺乏的。视频和代码可在此网址获得。