We consider the task of grasping a target object based on a natural language
command query. Previous work primarily focused on localizing the object given
the query, which requires a separate grasp detection module to grasp it. The
cascaded application of two pipelines incurs errors in overlapping multi-object
cases due to ambiguity in the individual outputs. This work proposes a model
named Command Grasping Network(CGNet) to directly output command satisficing
grasps from RGB image and textual command inputs. A dataset with ground truth
(image, command, grasps) tuple is generated based on the VMRD dataset to train
the proposed network. Experimental results on the generated test set show that
CGNet outperforms a cascaded object-retrieval and grasp detection baseline by a
large margin. Three physical experiments demonstrate the functionality and
performance of CGNet.

该论文提出了一种名为 CGNet 的模型，通过使用基于自然语言命令查询目标对象，直接从 RGB 图像和文本命令输入中输出命令满足的抓取。在基于 VMRD 数据集生成的基础上进行训练，该模型的实验结果比级联物体检索和抓取检测基线要优越很多，并通过三个物理实验证明了该模型的功能和性能。