This paper proposes Comprehensive Pathology Language Image Pre-training
(CPLIP), a new unsupervised technique designed to enhance the alignment of
images and text in histopathology for tasks such as classification and
segmentation. This methodology enriches vision-language models by leveraging
extensive data without needing ground truth annotations. CPLIP involves
constructing a pathology-specific dictionary, generating textual descriptions
for images using language models, and retrieving relevant images for each text
snippet via a pre-trained model. The model is then fine-tuned using a
many-to-many contrastive learning method to align complex interrelated concepts
across both modalities. Evaluated across multiple histopathology tasks, CPLIP
shows notable improvements in zero-shot learning scenarios, outperforming
existing methods in both interpretability and robustness and setting a higher
benchmark for the application of vision-language models in the field. To
encourage further research and replication, the code for CPLIP is available on
GitHub at this https URL

该研究提出了一种名为综合病理语言图像预训练（CPLIP）的无监督技术，用于增强组织病理学中图像和文本的对齐，以进行分类和分割等任务。CPLIP 通过利用大量数据而不需要地面真实注释，来丰富视觉 - 语言模型。通过构建特定于病理学的词典、使用语言模型为图像生成文本描述，并通过预训练模型检索每个文本片段的相关图像，再运用多对多对比学习方法对模型进行微调，以使复杂的相关概念在两种模态之间对齐。在多个组织病理学任务中进行评估，CPLIP 在零样本学习场景中显示出显著的改进，超越了现有方法在可解释性和鲁棒性方面，并在该领域中树立了更高的基准。为了鼓励进一步的研究和复制，CPLIP 的代码可在 GitHub 上获取。