Perceiving and manipulating 3D articulated objects (e.g., cabinets, doors) in
human environments is an important yet challenging task for future
home-assistant robots. The space of 3D articulated objects is exceptionally
rich in their myriad semantic categories, diverse shape geometry, and
complicated part functionality. Previous works mostly abstract kinematic
structure with estimated joint parameters and part poses as the visual
representations for manipulating 3D articulated objects. In this paper, we
propose object-centric actionable visual priors as a novel
perception-interaction handshaking point that the perception system outputs
more actionable guidance than kinematic structure estimation, by predicting
dense geometry-aware, interaction-aware, and task-aware visual action
affordance and trajectory proposals. We design an interaction-for-perception
framework VAT-Mart to learn such actionable visual representations by
simultaneously training a curiosity-driven reinforcement learning policy
exploring diverse interaction trajectories and a perception module summarizing
and generalizing the explored knowledge for pointwise predictions among diverse
shapes. Experiments prove the effectiveness of the proposed approach using the
large-scale PartNet-Mobility dataset in SAPIEN environment and show promising
generalization capabilities to novel test shapes, unseen object categories, and
real-world data. Project page: this https URL

本文提出了一种基于物体的行动视觉先验的方法，即预测密集的几何感知、交互感知和任务感知的行动特征，结合强化学习策略和感知模块进行训练，通过探索多样的交互轨迹和总结概括所探索的知识来实现更好的感知 - 交互效果，并在大规模数据集上验证了其有效性和推广能力。