We propose ImGeoNet, a multi-view image-based 3D object detection framework
that models a 3D space by an image-induced geometry-aware voxel representation.
Unlike previous methods which aggregate 2D features into 3D voxels without
considering geometry, ImGeoNet learns to induce geometry from multi-view images
to alleviate the confusion arising from voxels of free space, and during the
inference phase, only images from multiple views are required. Besides, a
powerful pre-trained 2D feature extractor can be leveraged by our
representation, leading to a more robust performance. To evaluate the
effectiveness of ImGeoNet, we conduct quantitative and qualitative experiments
on three indoor datasets, namely ARKitScenes, ScanNetV2, and ScanNet200. The
results demonstrate that ImGeoNet outperforms the current state-of-the-art
multi-view image-based method, ImVoxelNet, on all three datasets in terms of
detection accuracy. In addition, ImGeoNet shows great data efficiency by
achieving results comparable to ImVoxelNet with 100 views while utilizing only
40 views. Furthermore, our studies indicate that our proposed image-induced
geometry-aware representation can enable image-based methods to attain superior
detection accuracy than the seminal point cloud-based method, VoteNet, in two
practical scenarios: (1) scenarios where point clouds are sparse and noisy,
such as in ARKitScenes, and (2) scenarios involve diverse object classes,
particularly classes of small objects, as in the case in ScanNet200.

我们提出了 ImGeoNet，这是一个基于多视图图像的 3D 物体检测框架，通过图像诱导的几何感知体素表示对 3D 空间进行建模。ImGeoNet 通过学习从多视图图像中诱导几何感知来减轻体素空间中的混淆，并且在推理阶段只需要来自多视图的图像。此外，我们可以利用强大的预训练 2D 特征提取器来优化我们的表示，从而获得更强大的性能。通过在 ARKitScenes、ScanNetV2 和 ScanNet200 三个室内数据集上进行定量和定性实验，我们验证了 ImGeoNet 的有效性，并证明它在检测精度方面优于当前最先进的基于多视图图像的方法 ImVoxelNet。此外，ImGeoNet 在只利用 40 个视图的情况下，达到了与 ImVoxelNet 利用 100 个视图相当的结果，显示出较高的数据效率。此外，我们的研究表明，我们提出的图像诱导的几何感知表示可以使基于图像的方法在两种实际场景中实现比重点云方法 VoteNet 更高的检测准确性：（1）点云稀疏且嘈杂的场景，例如 ARKitScenes，以及（2）涉及多样化对象类别，特别是小对象类别的情况，例如 ScanNet200。