The use of deep reinforcement learning allows for high-dimensional state
descriptors, but little is known about how the choice of action representation
impacts the learning difficulty and the resulting performance. We compare the
impact of four different action parameterizations (torques, muscle-activations,
target joint angles, and target joint-angle velocities) in terms of learning
time, policy robustness, motion quality, and policy query rates. Our results
are evaluated on a gait-cycle imitation task for multiple planar articulated
figures and multiple gaits. We demonstrate that the local feedback provided by
higher-level action parameterizations can significantly impact the learning,
robustness, and quality of the resulting policies.

本文研究了用于高维状态描述的深度强化学习，并比较了不同的动作参数表示对学习难度、策略鲁棒性、运动质量和策略查询率的影响。从多个平面关节图形和多个步态的步态循环模仿任务的结果来看，本文表明较高级别的动作参数化所提供的本地反馈可以显著影响策略的学习、鲁棒性和质量。