Recently, virtual/pseudo-point-based 3D object detection that seamlessly
fuses RGB images and LiDAR data by depth completion has gained great attention.
However, virtual points generated from an image are very dense, introducing a
huge amount of redundant computation during detection. Meanwhile, noises
brought by inaccurate depth completion significantly degrade detection
precision. This paper proposes a fast yet effective backbone, termed
VirConvNet, based on a new operator VirConv (Virtual Sparse Convolution), for
virtual-point-based 3D object detection. VirConv consists of two key designs:
(1) StVD (Stochastic Voxel Discard) and (2) NRConv (Noise-Resistant Submanifold
Convolution). StVD alleviates the computation problem by discarding large
amounts of nearby redundant voxels. NRConv tackles the noise problem by
encoding voxel features in both 2D image and 3D LiDAR space. By integrating
VirConv, we first develop an efficient pipeline VirConv-L based on an early
fusion design. Then, we build a high-precision pipeline VirConv-T based on a
transformed refinement scheme. Finally, we develop a semi-supervised pipeline
VirConv-S based on a pseudo-label framework. On the KITTI car 3D detection test
leaderboard, our VirConv-L achieves 85% AP with a fast running speed of 56ms.
Our VirConv-T and VirConv-S attains a high-precision of 86.3% and 87.2% AP, and
currently rank 2nd and 1st, respectively. The code is available at
this https URL

本文提出了一种新的虚拟稀疏卷积操作算子 VirConv (Virtual Sparse Convolution)，并基于此设计了 VirConvNet，用于快速且有效地进行基于虚拟点的 3D 目标检测，并在 KITTI 3D car 检测中取得了领先的成果，代码公开。