This paper examines the robustness of a multi-modal computer vision model,
CLIP (Contrastive Language-Image Pretraining), in the context of unsupervised
learning. The main objective is twofold: first, to evaluate the robustness of
CLIP, and second, to explore strategies for augmenting its robustness. To
achieve this, we introduce a novel approach named LP-CLIP. This technique
involves the distillation of CLIP features through the incorporation of a
linear probing layer positioned atop its encoding structure. This newly added
layer is trained utilizing pseudo-labels produced by CLIP, coupled with a
self-training strategy. The LP-CLIP technique offers a promising approach to
enhance the robustness of CLIP without the need for annotations. By leveraging
a simple linear probing layer, we aim to improve the model's ability to
withstand various uncertainties and challenges commonly encountered in
real-world scenarios. Importantly, our approach does not rely on annotated
data, which makes it particularly valuable in situations where labeled data
might be scarce or costly to obtain. Our proposed approach increases the
robustness of CLIP with SOTA results compared to supervised technique on
various datasets.

利用 LP-CLIP 技术通过引入一个线性探测层来提高 CLIP 的鲁棒性，该技术利用 CLIP 生成的伪标签以及自训练策略进行训练，无需注释数据，能够增强模型在真实场景中应对多种不确定性和挑战的能力，并在各种数据集上实现了 SOTA 结果