Despite substantial progress in 3D object detection, advanced 3D detectors
often suffer from heavy computation overheads. To this end, we explore the
potential of knowledge distillation (KD) for developing efficient 3D object
detectors, focusing on popular pillar- and voxel-based detectors.In the absence
of well-developed teacher-student pairs, we first study how to obtain student
models with good trade offs between accuracy and efficiency from the
perspectives of model compression and input resolution reduction. Then, we
build a benchmark to assess existing KD methods developed in the 2D domain for
3D object detection upon six well-constructed teacher-student pairs. Further,
we propose an improved KD pipeline incorporating an enhanced logit KD method
that performs KD on only a few pivotal positions determined by teacher
classification response, and a teacher-guided student model initialization to
facilitate transferring teacher model's feature extraction ability to students
through weight inheritance. Finally, we conduct extensive experiments on the
Waymo dataset. Our best performing model achieves $65.75\%$ LEVEL 2 mAPH,
surpassing its teacher model and requiring only $44\%$ of teacher flops. Our
most efficient model runs 51 FPS on an NVIDIA A100, which is $2.2\times$ faster
than PointPillar with even higher accuracy. Code is available at
https://github.com/CVMI-Lab/SparseKD.

本文章提出了一种使用知识蒸馏技术开发高效 3D 物体检测器的方法，并在输入分辨率降低和模型压缩方面发掘教师学生模型的差异，基于现有的 KD 方法建立了一个基准测试，提出了一种改进的 KD 流程，最后在 Waymo 数据集上进行了大量实验，实现了比教师模型更高的精度和更快的运行速度。