3D neural networks are widely used in real-world applications (e.g., AR/VR
headsets, self-driving cars). They are required to be fast and accurate;
however, limited hardware resources on edge devices make these requirements
rather challenging. Previous work processes 3D data using either voxel-based or
point-based neural networks, but both types of 3D models are not
hardware-efficient due to the large memory footprint and random memory access.
In this paper, we study 3D deep learning from the efficiency perspective. We
first systematically analyze the bottlenecks of previous 3D methods. We then
combine the best from point-based and voxel-based models together and propose a
novel hardware-efficient 3D primitive, Point-Voxel Convolution (PVConv). We
further enhance this primitive with the sparse convolution to make it more
effective in processing large (outdoor) scenes. Based on our designed 3D
primitive, we introduce 3D Neural Architecture Search (3D-NAS) to explore the
best 3D network architecture given a resource constraint. We evaluate our
proposed method on six representative benchmark datasets, achieving
state-of-the-art performance with 1.8-23.7x measured speedup. Furthermore, our
method has been deployed to the autonomous racing vehicle of MIT Driverless,
achieving larger detection range, higher accuracy and lower latency.

该研究通过结合基于点和基于体素的模型，提出一种硬件高效的三维卷积方法，并进一步融合 稀疏卷积，以更有效地处理大场景，从硬件效率的角度系统研究三维深度学习。在此基础上，引入三维神经架构搜索来探索最佳三维网络架构，并在六个典型基准数据集上评估其性能，实现 1.8-23.7 倍的加速，已应用于 MIT Driverless 无人驾驶车辆上，实现了更大的检测范围，更高的准确性和更低的延迟。

PVNAS: 3D 点 - 体素卷积神经结构搜索

PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution

Self-driving cars need to understand 3D scenes efficiently and accurately in
order to drive safely. Given the limited hardware resources, existing 3D
perception models are not able to recognize small instances (e.g., pedestrians,
cyclists) very well due to the low-resolution voxelization and aggressive
downsampling. To this end, we propose Sparse Point-Voxel Convolution (SPVConv),
a lightweight 3D module that equips the vanilla Sparse Convolution with the
high-resolution point-based branch. With negligible overhead, this point-based
branch is able to preserve the fine details even from large outdoor scenes. To
explore the spectrum of efficient 3D models, we first define a flexible
architecture design space based on SPVConv, and we then present 3D Neural
Architecture Search (3D-NAS) to search the optimal network architecture over
this diverse design space efficiently and effectively. Experimental results
validate that the resulting SPVNAS model is fast and accurate: it outperforms
the state-of-the-art MinkowskiNet by 3.3%, ranking 1st on the competitive
SemanticKITTI leaderboard. It also achieves 8x computation reduction and 3x
measured speedup over MinkowskiNet with higher accuracy. Finally, we transfer
our method to 3D object detection, and it achieves consistent improvements over
the one-stage detection baseline on KITTI.

提出了一种轻量级的 Sparse Point-Voxel Convolution 模块作为基础，通过基于 SPVConv 的灵活体系结构设计空间，实现了 3D Neural Architecture Search 以搜索高效而有效的网络体系结构，并将该方法应用于自动驾驶和 3D 目标检测领域，取得了较好的实验结果。