We present a hybrid-view-based knowledge distillation framework, termed
HVDistill, to guide the feature learning of a point cloud neural network with a
pre-trained image network in an unsupervised man- ner. By exploiting the
geometric relationship between RGB cameras and LiDAR sensors, the
correspondence between the two modalities based on both image- plane view and
bird-eye view can be established, which facilitates representation learning.
Specifically, the image-plane correspondences can be simply ob- tained by
projecting the point clouds, while the bird- eye-view correspondences can be
achieved by lifting pixels to the 3D space with the predicted depths un- der
the supervision of projected point clouds. The image teacher networks provide
rich semantics from the image-plane view and meanwhile acquire geometric
information from the bird-eye view. Indeed, image features from the two views
naturally comple- ment each other and together can ameliorate the learned
feature representation of the point cloud stu- dent networks. Moreover, with a
self-supervised pre- trained 2D network, HVDistill requires neither 2D nor 3D
annotations. We pre-train our model on nuScenes dataset and transfer it to
several downstream tasks on nuScenes, SemanticKITTI, and KITTI datasets for
evaluation. Extensive experimental results show that our method achieves
consistent improvements over the baseline trained from scratch and
significantly out- performs the existing schemes. Codes are available at
git@github.com:zhangsha1024/HVDistill.git.

基于混合视图的知识蒸馏 (HVDistill) 框架用于指导点云神经网络的特征学习，通过利用 RGB 相机和 LiDAR 传感器之间的几何关系，在无监督的方式下建立了两种模态之间的对应关系，其中基于图像平面视图和鸟瞰图可以实现，这有助于表示学习。