3D object detection task from lidar or camera sensors is essential for autonomous driving. Pioneer attempts at multi-modality fusion complement the sparse lidar point clouds with rich semantic texture information from images at the cost of extra network designs and overhead. In this work, we propose a novel semantic passing framework, named SPNet, to boost the performance of existing lidar-based 3D detection models with the guidance of rich context painting, with no extra computation cost during inference. Our key design is to first exploit the potential instructive semantic knowledge within the ground-truth labels by training a semantic-painted teacher model and then guide the pure-lidar network to learn the semantic-painted representation via knowledge passing modules at different granularities: class-wise passing, pixel-wise passing and instance-wise passing. Experimental results show that the proposed SPNet can seamlessly cooperate with most existing 3D detection frameworks with 1~5% AP gain and even achieve new state-of-the-art 3D detection performance on the KITTI test benchmark. Code is available at: https://github.com/jb892/SPNet.

本文提出了一种名为SPNet的新型语义传递框架，它使用丰富的上下文绘制来提高现有基于lidar的3D检测模型的性能，通过知识传递模块在不同的粒度上指导纯lidar网络学习经过语义渲染的表示，可以与大多数现有的3D检测框架无缝协作，在KITTI测试基准上甚至实现了新的3D检测性能最佳水平。

使用语义通信网络提升 3D 物体检测的绘制与精简方法