6-DoF robotic grasping is a long-lasting but unsolved problem. Recent methods
utilize strong 3D networks to extract geometric grasping representations from
depth sensors, demonstrating superior accuracy on common objects but perform
unsatisfactorily on photometrically challenging objects, e.g., objects in
transparent or reflective materials. The bottleneck lies in that the surface of
these objects can not reflect back accurate depth due to the absorption or
refraction of light. In this paper, in contrast to exploiting the inaccurate
depth data, we propose the first RGB-only 6-DoF grasping pipeline called
MonoGraspNet that utilizes stable 2D features to simultaneously handle
arbitrary object grasping and overcome the problems induced by photometrically
challenging objects. MonoGraspNet leverages keypoint heatmap and normal map to
recover the 6-DoF grasping poses represented by our novel representation
parameterized with 2D keypoints with corresponding depth, grasping direction,
grasping width, and angle. Extensive experiments in real scenes demonstrate
that our method can achieve competitive results in grasping common objects and
surpass the depth-based competitor by a large margin in grasping
photometrically challenging objects. To further stimulate robotic manipulation
research, we additionally annotate and open-source a multi-view and multi-scene
real-world grasping dataset, containing 120 objects of mixed photometric
complexity with 20M accurate grasping labels.

本文提出了首个基于 RGB 图像的 6 自由度机器人抓取方法 MonoGraspNet，通过利用稳定的 2D 特征以及关键点热度图和法向量图来同时解决处理透明或反射材料等光学难题，实验表明我们的方法在抓取常见物体方面能够达到有竞争性的结果，并在在抓取光学难题物体方面领先于深度学习方法，此外我们也公开了一个包含 120 个物品的多视角多场景数据集和 2000 万的准确抓取标签。