Modular Reinforcement Learning (RL) decentralizes the control of multi-joint robots by learning policies for each actuator. Previous work on modular RL has proven its ability to control morphologically different agents with a shared actuator policy. However, with the increase in the Degree of Freedom (DoF) of robots, training a morphology-generalizable modular controller becomes exponentially difficult. Motivated by the way the human central nervous system controls numerous muscles, we propose a Synergy-Oriented LeARning (SOLAR) framework that exploits the redundant nature of DoF in robot control. Actuators are grouped into synergies by an unsupervised learning method, and a synergy action is learned to control multiple actuators in synchrony. In this way, we achieve a low-rank control at the synergy level. We extensively evaluate our method on a variety of robot morphologies, and the results show its superior efficiency and generalizability, especially on robots with a large DoF like Humanoids++ and UNIMALs.

本研究提出了一种基于协同作用的学习（SOLAR）框架，利用机器人控制中度自由度的冗余特性，将执行器分为若干个协同作用组，并学习一种协同作用动作来同步控制多个执行器。通过这种方式，在协同作用水平上实现低秩控制，其可有效地控制机器人的运动。在机器人形态多样化的数据集上取得了良好的控制效果。

基于肌肉协同作用的低秩模块化强化学习