3D visual grounding involves finding a target object in a 3D scene that corresponds to a given sentence query. Although many approaches have been proposed and achieved impressive performance, they all require dense object-sentence pair annotations in 3D point clouds, which are both time-consuming and expensive. To address the problem that fine-grained annotated data is difficult to obtain, we propose to leverage weakly supervised annotations to learn the 3D visual grounding model, i.e., only coarse scene-sentence correspondences are used to learn object-sentence links. To accomplish this, we design a novel semantic matching model that analyzes the semantic similarity between object proposals and sentences in a coarse-to-fine manner. Specifically, we first extract object proposals and coarsely select the top-K candidates based on feature and class similarity matrices. Next, we reconstruct the masked keywords of the sentence using each candidate one by one, and the reconstructed accuracy finely reflects the semantic similarity of each candidate to the query. Additionally, we distill the coarse-to-fine semantic matching knowledge into a typical two-stage 3D visual grounding model, which reduces inference costs and improves performance by taking full advantage of the well-studied structure of the existing architectures. We conduct extensive experiments on ScanRefer, Nr3D, and Sr3D, which demonstrate the effectiveness of our proposed method.

利用弱监督注释学习3D视觉定位模型，通过分析对象提案和句子之间的语义相似性，我们构建了一个语义匹配模型，将粗粒度的场景-句子对应关系用于学习目标-句子链接。同时，我们将粗精度的语义匹配知识融入经典的两阶段3D视觉定位模型，以减少推理成本并提高性能。通过在ScanRefer、Nr3D和Sr3D上进行广泛实验证明了我们提出方法的有效性。

蒸馏粗粒到精粒语义匹配知识用于弱监督的三维视觉接地