The ability to understand the ways to interact with objects from visual cues, a.k.a. visual affordance, is essential to vision-guided robotic research. This involves categorizing, segmenting and reasoning of visual affordance. Relevant studies in 2D and 2.5D image domains have been made previously, however, a truly functional understanding of object affordance requires learning and prediction in the 3D physical domain, which is still absent in the community. In this work, we present a 3D AffordanceNet dataset, a benchmark of 23k shapes from 23 semantic object categories, annotated with 18 visual affordance categories. Based on this dataset, we provide three benchmarking tasks for evaluating visual affordance understanding, including full-shape, partial-view and rotation-invariant affordance estimations. Three state-of-the-art point cloud deep learning networks are evaluated on all tasks. In addition we also investigate a semi-supervised learning setup to explore the possibility to benefit from unlabeled data. Comprehensive results on our contributed dataset show the promise of visual affordance understanding as a valuable yet challenging benchmark.

本论文介绍了一个名为 3D AffordanceNet 的数据集，其中包含来自 23 个语义对象类别的 23k 个形状，注释了 18 个视觉可用性类别，并提供了三个评估视觉可用性理解的基准测试任务。作者评估了三种最先进的点云深度学习网络，并研究了半监督学习设置探索利用未标记数据的可能性。综合结果表明，视觉可用性理解是一个有价值但具有挑战性的基准测试。

3D AffordanceNet：视觉物体可供性理解的基准测试