Dynamic tasks like table tennis are relatively easy to learn for humans but
pose significant challenges to robots. Such tasks require accurate control of
fast movements and precise timing in the presence of imprecise state estimation
of the flying ball and the robot. Reinforcement Learning (RL) has shown promise
in learning of complex control tasks from data. However, applying step-based RL
to dynamic tasks on real systems is safety-critical as RL requires exploring
and failing safely for millions of time steps in high-speed regimes. In this
paper, we demonstrate that safe learning of table tennis using model-free
Reinforcement Learning can be achieved by using robot arms driven by pneumatic
artificial muscles (PAMs). Softness and back-drivability properties of PAMs
prevent the system from leaving the safe region of its state space. In this
manner, RL empowers the robot to return and smash real balls with 5 m\s and
12m\s on average to a desired landing point. Our setup allows the agent to
learn this safety-critical task (i) without safety constraints in the
algorithm, (ii) while maximizing the speed of returned balls directly in the
reward function (iii) using a stochastic policy that acts directly on the
low-level controls of the real system and (iv) trains for thousands of trials
(v) from scratch without any prior knowledge. Additionally, we present HYSR, a
practical hybrid sim and real training that avoids playing real balls during
training by randomly replaying recorded ball trajectories in simulation and
applying actions to the real robot. This work is the first to (a) fail-safe
learn of a safety-critical dynamic task using anthropomorphic robot arms, (b)
learn a precision-demanding problem with a PAM-driven system despite the
control challenges and (c) train robots to play table tennis without real
balls. Videos and datasets are available at muscularTT.embodied.ml.

本文介绍一种利用气动人工肌肉和强化学习来安全地训练机器人学习打乒乓球的方法，同时演示了这种方法可以让机器人在高速运动下实现回球和攻击