The low-level sensory and motor signals in deep reinforcement learning, which
exist in high-dimensional spaces such as image observations or motor torques,
are inherently challenging to understand or utilize directly for downstream
tasks. While sensory representations have been extensively studied, the
representations of motor actions are still an area of active exploration. Our
work reveals that a space containing meaningful action representations emerges
when a multi-task policy network takes as inputs both states and task
embeddings. Moderate constraints are added to improve its representation
ability. Therefore, interpolated or composed embeddings can function as a
high-level interface within this space, providing instructions to the agent for
executing meaningful action sequences. Empirical results demonstrate that the
proposed action representations are effective for intra-action interpolation
and inter-action composition with limited or no additional learning.
Furthermore, our approach exhibits superior task adaptation ability compared to
strong baselines in Mujoco locomotion tasks. Our work sheds light on the
promising direction of learning action representations for efficient,
adaptable, and composable RL, forming the basis of abstract action planning and
the understanding of motor signal space. Project page:
this https URL

该论文研究了深度强化学习中低级感知和运动信号的表示方法，提出通过多任务策略网络输入状态和任务嵌入的方法得到有意义的运动表示空间，并在此基础上进行高级别指令的执行规划，实验结果表明该方法优于现有强基线方法，具有较强的任务适应能力。