Self-supervised image networks can be used to address complex 2D tasks (e.g.,
semantic segmentation, object discovery) very efficiently and with little or no
downstream supervision. However, self-supervised 3D networks on lidar data do
not perform as well for now. A few methods therefore propose to distill
high-quality self-supervised 2D features into 3D networks. The most recent ones
doing so on autonomous driving data show promising results. Yet, a performance
gap persists between these distilled features and fully-supervised ones. In
this work, we revisit 2D-to-3D distillation. First, we propose, for semantic
segmentation, a simple approach that leads to a significant improvement
compared to prior 3D distillation methods. Second, we show that distillation in
high capacity 3D networks is key to reach high quality 3D features. This
actually allows us to significantly close the gap between unsupervised
distilled 3D features and fully-supervised ones. Last, we show that our
high-quality distilled representations can also be used for open-vocabulary
segmentation and background/foreground discovery.

自监督图像网络在解决复杂的 2D 任务（如语义分割、目标发现）时非常高效且几乎没有下游监督要求，然而，当前基于激光雷达数据的自监督 3D 网络表现不佳，因此有几种方法提议将高质量的自监督 2D 特征转移到 3D 网络中，最近在自动驾驶数据上进行的尝试显示了有希望的结果，然而，这些转移后的特征与完全监督的特征之间仍然存在差距，本文重新审视了 2D 到 3D 转移，首先，针对语义分割，我们提出了一种简单的方法，相对于之前的 3D 转移方法实现了显著的提高，其次，我们证明了在高容量的 3D 网络中进行转移对于获得高质量的 3D 特征至关重要，这实际上使我们能够显著缩小无监督转移的 3D 特征与完全监督特征之间的差距，最后，我们表明我们所得到的高质量转移表示还可以用于开放词汇的分割和背景 / 前景发现。