3D visual grounding involves finding a target object in a 3D scene that
corresponds to a given sentence query. Although many approaches have been
proposed and achieved impressive performance, they all require dense
object-sentence pair annotations in 3D point clouds, which are both
time-consuming and expensive. To address the problem that fine-grained
annotated data is difficult to obtain, we propose to leverage weakly supervised
annotations to learn the 3D visual grounding model, i.e., only coarse
scene-sentence correspondences are used to learn object-sentence links. To
accomplish this, we design a novel semantic matching model that analyzes the
semantic similarity between object proposals and sentences in a coarse-to-fine
manner. Specifically, we first extract object proposals and coarsely select the
top-K candidates based on feature and class similarity matrices. Next, we
reconstruct the masked keywords of the sentence using each candidate one by
one, and the reconstructed accuracy finely reflects the semantic similarity of
each candidate to the query. Additionally, we distill the coarse-to-fine
semantic matching knowledge into a typical two-stage 3D visual grounding model,
which reduces inference costs and improves performance by taking full advantage
of the well-studied structure of the existing architectures. We conduct
extensive experiments on ScanRefer, Nr3D, and Sr3D, which demonstrate the
effectiveness of our proposed method.

利用弱监督注释学习 3D 视觉定位模型，通过分析对象提案和句子之间的语义相似性，我们构建了一个语义匹配模型，将粗粒度的场景 - 句子对应关系用于学习目标 - 句子链接。同时，我们将粗精度的语义匹配知识融入经典的两阶段 3D 视觉定位模型，以减少推理成本并提高性能。通过在 ScanRefer、Nr3D 和 Sr3D 上进行广泛实验证明了我们提出方法的有效性。

蒸馏粗粒到精粒语义匹配知识用于弱监督的三维视觉接地

Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly  Supervised 3D Visual Grounding

We introduce HoVer (HOppy VERification), a dataset for many-hop evidence
extraction and fact verification. It challenges models to extract facts from
several Wikipedia articles that are relevant to a claim and classify whether
the claim is Supported or Not-Supported by the facts. In HoVer, the claims
require evidence to be extracted from as many as four English Wikipedia
articles and embody reasoning graphs of diverse shapes. Moreover, most of the
3/4-hop claims are written in multiple sentences, which adds to the complexity
of understanding long-range dependency relations such as coreference. We show
that the performance of an existing state-of-the-art semantic-matching model
degrades significantly on our dataset as the number of reasoning hops
increases, hence demonstrating the necessity of many-hop reasoning to achieve
strong results. We hope that the introduction of this challenging dataset and
the accompanying evaluation task will encourage research in many-hop fact
retrieval and information verification. We make the HoVer dataset publicly
available at this https URL

介绍了 HoVer 数据集，用于测试模型在从多篇维基百科文章中提取证据并分类核实主张上的表现，展示了许多跳推理的必要性和现有语义匹配模型在推理跳数增加时性能下降的现象。