We study the choice of action space in robot manipulation learning and
sim-to-real transfer. We define metrics that assess the performance, and
examine the emerging properties in the different action spaces. We train over
250 reinforcement learning~(RL) agents in simulated reaching and pushing tasks,
using 13 different control spaces. The choice of action spaces spans popular
choices in the literature as well as novel combinations of common design
characteristics. We evaluate the training performance in simulation and the
transfer to a real-world environment. We identify good and bad characteristics
of robotic action spaces and make recommendations for future designs. Our
findings have important implications for the design of RL algorithms for robot
manipulation tasks, and highlight the need for careful consideration of action
spaces when training and transferring RL agents for real-world robotics.

我们研究了机器人操作学习和从仿真到实际的转移中的行动空间选择。我们定义了评估性能的度量标准，并研究了不同行动空间的新兴特性。我们使用 13 种不同的控制空间，在模拟的抓取和推动任务中训练了超过 250 个强化学习代理。行动空间的选择涵盖了文献中的热门选择以及常见设计特征的新组合。我们对仿真中的训练性能以及向实际环境的转移进行了评估。我们确认了机器人行动空间的优点和缺点，并对未来设计提出了建议。我们的发现对于机器人操作任务的强化学习算法设计具有重要意义，并强调了在训练和转移强化学习代理时对行动空间的慎重考虑的必要性。