We address adversarial attacks on the actuators at the joints of legged
robots trained by deep reinforcement learning. The vulnerability to the joint
attacks can significantly impact the safety and robustness of legged robots. In
this study, we demonstrate that the adversarial perturbations to the torque
control signals of the actuators can significantly reduce the rewards and cause
walking instability in robots. To find the adversarial torque perturbations, we
develop black-box adversarial attacks, where, the adversary cannot access the
neural networks trained by deep reinforcement learning. The black box attack
can be applied to legged robots regardless of the architecture and algorithms
of deep reinforcement learning. We employ three search methods for the
black-box adversarial attacks: random search, differential evolution, and
numerical gradient descent methods. In experiments with the quadruped robot
Ant-v2 and the bipedal robot Humanoid-v2, in OpenAI Gym environments, we find
that differential evolution can efficiently find the strongest torque
perturbations among the three methods. In addition, we realize that the
quadruped robot Ant-v2 is vulnerable to the adversarial perturbations, whereas
the bipedal robot Humanoid-v2 is robust to the perturbations. Consequently, the
joint attacks can be used for proactive diagnosis of robot walking instability.

该研究针对被深度强化学习训练的腿式机器人关节执行器的对抗攻击进行了探讨，发现对执行器扭矩控制信号的对抗扰动会显著降低机器人的奖励并导致行走不稳定。作者使用黑盒对抗攻击策略，采用三种搜索方法在 OpenAI Gym 的环境下对四足机器人 Ant-v2 和双足机器人 Humanoid-v2 进行实验，证明差分进化方法可以高效地找到最强的扭矩扰动，并发现四足机器人 Ant-v2 容易受到对抗攻击的影响，而双足机器人 Humanoid-v2 相对稳定。因此，这种关节攻击可用于机器人步态不稳定的主动诊断。