Integrating learning-based techniques, especially reinforcement learning,
into robotics is promising for solving complex problems in unstructured
environments. However, most existing approaches are trained in well-tuned
simulators and subsequently deployed on real robots without online fine-tuning.
In this setting, the simulation's realism seriously impacts the deployment's
success rate. Instead, learning with real-world interaction data offers a
promising alternative: not only eliminates the need for a fine-tuned simulator
but also applies to a broader range of tasks where accurate modeling is
unfeasible. One major problem for on-robot reinforcement learning is ensuring
safety, as uncontrolled exploration can cause catastrophic damage to the robot
or the environment. Indeed, safety specifications, often represented as
constraints, can be complex and non-linear, making safety challenging to
guarantee in learning systems. In this paper, we show how we can impose complex
safety constraints on learning-based robotics systems in a principled manner,
both from theoretical and practical points of view. Our approach is based on
the concept of the Constraint Manifold, representing the set of safe robot
configurations. Exploiting differential geometry techniques, i.e., the tangent
space, we can construct a safe action space, allowing learning agents to sample
arbitrary actions while ensuring safety. We demonstrate the method's
effectiveness in a real-world Robot Air Hockey task, showing that our method
can handle high-dimensional tasks with complex constraints. Videos of the real
robot experiments are available on the project website
(this https URL).

将基于学习的技术，特别是强化学习，整合到机器人技术中来解决非结构化环境中的复杂问题很具有前景。本文展示了我们如何以原则性的方式将复杂的安全约束强加于基于学习的机器人系统中，从理论和实践的角度进行了展示。