Estimating robot pose from RGB images is a crucial problem in computer vision
and robotics. While previous methods have achieved promising performance, most
of them presume full knowledge of robot internal states, e.g. ground-truth
robot joint angles, which are not always available in real-world scenarios. On
the other hand, existing approaches that estimate robot pose without joint
state priors suffer from heavy computation burdens and thus cannot support
real-time applications. This work addresses the urgent need for efficient robot
pose estimation with unknown states. We propose an end-to-end pipeline for
real-time, holistic robot pose estimation from a single RGB image, even in the
absence of known robot states. Our method decomposes the problem into
estimating camera-to-robot rotation, robot state parameters, keypoint
locations, and root depth. We further design a corresponding neural network
module for each task. This approach allows for learning multi-facet
representations and facilitates sim-to-real transfer through self-supervised
learning. Notably, our method achieves inference with a single feedforward,
eliminating the need for costly test-time iterative optimization. As a result,
it delivers a 12-time speed boost with state-of-the-art accuracy, enabling
real-time holistic robot pose estimation for the first time. Code is available
at this https URL

通过单个 RGB 图像进行实时机器人姿态估计的端到端流程，无需已知机器人状态，并达到了领先水平的 12 倍速度提升。