In 3D Referring Expression Segmentation (3D-RES), the earlier approach adopts
a two-stage paradigm, extracting segmentation proposals and then matching them
with referring expressions. However, this conventional paradigm encounters
significant challenges, most notably in terms of the generation of lackluster
initial proposals and a pronounced deceleration in inference speed. Recognizing
these limitations, we introduce an innovative end-to-end Superpoint-Text
Matching Network (3D-STMN) that is enriched by dependency-driven insights. One
of the keystones of our model is the Superpoint-Text Matching (STM) mechanism.
Unlike traditional methods that navigate through instance proposals, STM
directly correlates linguistic indications with their respective superpoints,
clusters of semantically related points. This architectural decision empowers
our model to efficiently harness cross-modal semantic relationships, primarily
leveraging densely annotated superpoint-text pairs, as opposed to the more
sparse instance-text pairs. In pursuit of enhancing the role of text in guiding
the segmentation process, we further incorporate the Dependency-Driven
Interaction (DDI) module to deepen the network's semantic comprehension of
referring expressions. Using the dependency trees as a beacon, this module
discerns the intricate relationships between primary terms and their associated
descriptors in expressions, thereby elevating both the localization and
segmentation capacities of our model. Comprehensive experiments on the
ScanRefer benchmark reveal that our model not only set new performance
standards, registering an mIoU gain of 11.7 points but also achieve a
staggering enhancement in inference speed, surpassing traditional methods by
95.7 times. The code and models are available at
this https URL

3D-RES 中，采用了一个创新的端到端 Superpoint-Text Matching Network (3D-STMN)，该网络通过依赖驱动的洞察力丰富而充实。我们的模型具有超点文本匹配机制，该机制通过将语言指示与其对应的类别超点直接相关联，实现了跨模态语义关系的高效利用，并且进一步引入了依赖驱动交互模块来提升模型对指代表达的语义理解，从而提升模型的本地化和分割能力。在 ScanRefer 基准上的综合实验结果表明，我们的模型不仅取得了新的性能标准，在 mIoU 方面提升了 11.7 个百分点，而且速度提升惊人，超过传统方法 95.7 倍。