Recent advancements in 3D perception systems have significantly improved
their ability to perform visual recognition tasks such as segmentation.
However, these systems still heavily rely on explicit human instruction to
identify target objects or categories, lacking the capability to actively
reason and comprehend implicit user intentions. We introduce a novel
segmentation task known as reasoning part segmentation for 3D objects, aiming
to output a segmentation mask based on complex and implicit textual queries
about specific parts of a 3D object. To facilitate evaluation and benchmarking,
we present a large 3D dataset comprising over 60k instructions paired with
corresponding ground-truth part segmentation annotations specifically curated
for reasoning-based 3D part segmentation. We propose a model that is capable of
segmenting parts of 3D objects based on implicit textual queries and generating
natural language explanations corresponding to 3D object segmentation requests.
Experiments show that our method achieves competitive performance to models
that use explicit queries, with the additional abilities to identify part
concepts, reason about them, and complement them with world knowledge. Our
source code, dataset, and trained models are available at
this https URL

鉴于目标物体识别需求，我们介绍了一种称为 3D 物体推理部分分割的新型细分任务，旨在根据复杂和隐含的文本查询输出一个基于 3D 物体特定部分的分割蒙版，并提出了一种能够分割 3D 物体部分的模型，并生成与 3D 物体分割请求相对应的自然语言解释。