Humans are able to perform a myriad of sophisticated tasks by drawing upon
skills acquired through prior experience. For autonomous agents to have this
capability, they must be able to extract reusable skills from past experience
that can be recombined in new ways for subsequent tasks. Furthermore, when
controlling complex high-dimensional morphologies, such as humanoid bodies,
tasks often require coordination of multiple skills simultaneously. Learning
discrete primitives for every combination of skills quickly becomes
prohibitive. Composable primitives that can be recombined to create a large
variety of behaviors can be more suitable for modeling this combinatorial
explosion. In this work, we propose multiplicative compositional policies
(MCP), a method for learning reusable motor skills that can be composed to
produce a range of complex behaviors. Our method factorizes an agent's skills
into a collection of primitives, where multiple primitives can be activated
simultaneously via multiplicative composition. This flexibility allows the
primitives to be transferred and recombined to elicit new behaviors as
necessary for novel tasks. We demonstrate that MCP is able to extract
composable skills for highly complex simulated characters from pre-training
tasks, such as motion imitation, and then reuse these skills to solve
challenging continuous control tasks, such as dribbling a soccer ball to a
goal, and picking up an object and transporting it to a target location.

本研究提出了一种用于学习可重复使用的运动技能并将其组合为复杂行为的方法，该方法称为 “可组合的原语”，可以将代理人的技能分解为基本元素，这些基本元素可以通过乘法组合同时激活，从而使基本元素能够相互传输和重组，以适应新任务的需要，并演示了该方法在模拟环境中提取可组合的技能并重用这些技能来解决连续控制任务的能力。