It is crucial to address the following issues for ubiquitous robotics
manipulation applications: (a) vision-based manipulation tasks require the
robot to visually learn and understand the object with rich information like
dense object descriptors; and (b) sim-to-real transfer in robotics aims to
close the gap between simulated and real data. In this paper, we present
Sim-to-Real Dense Object Nets (SRDONs), a dense object descriptor that not only
understands the object via appropriate representation but also maps simulated
and real data to a unified feature space with pixel consistency. We proposed an
object-to-object matching method for image pairs from different scenes and
different domains. This method helps reduce the effort of training data from
real-world by taking advantage of public datasets, such as GraspNet. With
sim-to-real object representation consistency, our SRDONs can serve as a
building block for a variety of sim-to-real manipulation tasks. We demonstrate
in experiments that pre-trained SRDONs significantly improve performances on
unseen objects and unseen visual environments for various robotic tasks with
zero real-world training.

本研究提出了一种名为 Sim-to-Real Dense Object Nets (SRDONs) 的密集对象描述符，可实现从虚拟到真实环境的机器人物体识别一致性，并能够映射到一个统一的特征空间。在实验中，我们证明预训练的 SRDONs 可大幅提高各种机器人任务中对未知物体和未知视觉环境的性能表现。