Despite that deep learning methods have achieved remarkable performance in
pathology image classification, they heavily rely on labeled data, demanding
extensive human annotation efforts. In this study, we present a novel human
annotation-free method for pathology image classification by leveraging
pre-trained Vision-Language Models (VLMs). Without human annotation, pseudo
labels of the training set are obtained by utilizing the zero-shot inference
capabilities of VLM, which may contain a lot of noise due to the domain shift
between the pre-training data and the target dataset. To address this issue, we
introduce VLM-CPL, a novel approach based on consensus pseudo labels that
integrates two noisy label filtering techniques with a semi-supervised learning
strategy. Specifically, we first obtain prompt-based pseudo labels with
uncertainty estimation by zero-shot inference with the VLM using multiple
augmented views of an input. Then, by leveraging the feature representation
ability of VLM, we obtain feature-based pseudo labels via sample clustering in
the feature space. Prompt-feature consensus is introduced to select reliable
samples based on the consensus between the two types of pseudo labels. By
rejecting low-quality pseudo labels, we further propose High-confidence Cross
Supervision (HCS) to learn from samples with reliable pseudo labels and the
remaining unlabeled samples. Experimental results showed that our method
obtained an accuracy of 87.1% and 95.1% on the HPH and LC25K datasets,
respectively, and it largely outperformed existing zero-shot classification and
noisy label learning methods. The code is available at
this https URL

通过利用预训练的视觉语言模型，以无人工注释的方式进行病理图像分类，通过众包伪标签技术获得伪标签并通过一致性筛选以优化准确性。