This study tackles the representative yet challenging contact-rich peg-in-hole task of robotic assembly, using a soft wrist that can operate more safely and tolerate lower-frequency control signals than a rigid one. Previous studies often use a fully observable formulation, requiring external setups or estimators for the peg-to-hole pose. In contrast, we use a partially observable formulation and deep reinforcement learning from demonstrations to learn a memory-based agent that acts purely on haptic and proprioceptive signals. Moreover, previous works do not incorporate potential domain symmetry and thus must search for solutions in a bigger space. Instead, we propose to leverage the symmetry for sample efficiency by augmenting the training data and constructing auxiliary losses to force the agent to adhere to the symmetry. Results in simulation with five different symmetric peg shapes show that our proposed agent can be comparable to or even outperform a state-based agent. In particular, the sample efficiency also allows us to learn directly on the real robot within 3 hours.

这项研究使用软手腕来解决机器人装配中具有代表性且具有挑战性的插入孔位任务，其可以比刚性手腕更安全操作且容忍较低频率的控制信号。我们使用部分可观察的形式并通过展示学习和基于记忆的深度强化学习来训练一个完全基于触觉和本体感知信号行动的智能体。此外，我们利用潜在的领域对称性以提高样本效率，并通过构建辅助损失推动智能体遵守对称性。在模拟环境中，我们的智能体在五个不同的对称插头形状上显示出与基于状态的智能体相当甚至更好的性能。特别是，样本效率还使我们能够在3小时内直接在真实机器人上学习。

对称感知强化学习在部分可观测条件下软手腕机器人装配中的应用