Vision-based teleoperation offers the possibility to endow robots with
human-level intelligence to physically interact with the environment, while
only requiring low-cost camera sensors. However, current vision-based
teleoperation systems are designed and engineered towards a particular robot
model and deploy environment, which scales poorly as the pool of the robot
models expands and the variety of the operating environment increases. In this
paper, we propose AnyTeleop, a unified and general teleoperation system to
support multiple different arms, hands, realities, and camera configurations
within a single system. Although being designed to provide great flexibility to
the choice of simulators and real hardware, our system can still achieve great
performance. For real-world experiments, AnyTeleop can outperform a previous
system that was designed for a specific robot hardware with a higher success
rate, using the same robot. For teleoperation in simulation, AnyTeleop leads to
better imitation learning performance, compared with a previous system that is
particularly designed for that simulator. Project page: this http URL

提出了一个统一而通用的遥操作系统 AnyTeleop，支持多个不同的机器人模型、手型、现实以及摄像头配置，实现了高性能的真实世界实验和仿真中的模仿学习性能。