Perceiving and manipulating objects in a generalizable way has been actively studied by the computer vision and robotics communities, where cross-category generalizable manipulation skills are highly desired yet underexplored. In this work, we propose to learn such generalizable perception and manipulation via Generalizable and Actionable Parts (GAParts). By identifying and defining 9 GAPart classes (e.g. buttons, handles, etc), we show that our part-centric approach allows our method to learn object perception and manipulation skills from seen object categories and directly generalize to unseen categories. Following the GAPart definition, we construct a large-scale part-centric interactive dataset, GAPartNet, where rich, part-level annotations (semantics, poses) are provided for 1166 objects and 8489 part instances. Based on GAPartNet, we investigate three cross-category tasks: part segmentation, part pose estimation, and part-based object manipulation. Given the large domain gaps between seen and unseen object categories, we propose a strong 3D segmentation method from the perspective of domain generalization by integrating adversarial learning techniques. Our method outperforms all existing methods by a large margin, no matter on seen or unseen categories. Furthermore, with part segmentation and pose estimation results, we leverage the GAPart pose definition to design part-based manipulation heuristics that can generalize well to unseen object categories in both simulation and real world. The dataset and code will be released.

通过 Generalizable and Actionable Parts (GAParts) 来实现交叉类别通用的目标认知和操作，构建大规模的基于部件的交互数据集 GAPartNet 并利用对抗学习技术提出鲁棒的三维分割方法，通过部分分割和姿态估计结果结合 GAPart 的姿态定义设计出一些基于部件的操作启发式策略能普遍适用于模拟器和现实世界的未知物体类别，方法在不同类别上大幅优于现有方法。

GAPartNet: 通过泛化可操作部件实现跨类别通用物体感知与操作