Besides image classification, Contrastive Language-Image Pre-training (CLIP)
has accomplished extraordinary success for a wide range of vision tasks,
including object-level and 3D space understanding. However, it's still
challenging to transfer semantic knowledge learned from CLIP into more
intricate tasks of quantified targets, such as depth estimation with geometric
information. In this paper, we propose to apply CLIP for zero-shot monocular
depth estimation, named DepthCLIP. We found that the patches of the input image
could respond to a certain semantic distance token and then be projected to a
quantified depth bin for coarse estimation. Without any training, our DepthCLIP
surpasses existing unsupervised methods and even approaches the early
fully-supervised networks. To our best knowledge, we are the first to conduct
zero-shot adaptation from the semantic language knowledge to quantified
downstream tasks and perform zero-shot monocular depth estimation. We hope our
work could cast a light on future research. The code is available at
this https URL.

本文提出一种名为 DepthCLIP 的方法，将基于对比学习的语言 - 图像预训练（CLIP）应用于零样本单目深度估计任务中，成功地将语义上的知识迁移到了更为复杂的几何量化目标中，而无需训练，超越了现有的无监督方法，甚至接近了早期的全监督网络。