Contrastive language-image Pre-training (CLIP) [13] can leverage large
datasets of unlabeled Image-Text pairs, which have demonstrated impressive
performance in various downstream tasks. Given that annotating medical data is
time-consuming and laborious, Image-Text Pre-training has promising
applications in exploiting large-scale medical image and radiology report
datasets. However, medical Image-Text Pre-training faces several challenges, as
follows: (1) Due to privacy concerns, the amount of available medical data is
relatively small compared to natural data, leading to weaker generalization
ability of the model. (2) Medical images are highly similar with only
fine-grained differences in subtleties, resulting in a large number of
false-negative sample pairs in comparison learning. (3) The hand-crafted Prompt
usually differs from the natural medical image report, Subtle changes in
wording can lead to significant differences in performance. In this paper, we
propose a unified Image-Text-Label contrastive learning framework based on
continuous prompts, with three main contributions. First, We unified the data
of images, text, and labels, which greatly expanded the training data that the
model could utilize. Second, we address the issue of data diversity and the
impact of hand-crafted prompts on model performance by introducing continuous
implicit prompts. Lastly, we propose a ImageText-Label contrastive Training to
mitigate the problem of too many false-negative samples. We demonstrate through
sufficient experiments that the Unified Medical Contrastive Learning (UMCL)
framework exhibits excellent performance on several downstream tasks.

本文提出了一种基于连续提示的统一图像 - 文本 - 标签对比学习框架，从数据统一性，多样性和假阴性样本等方面解决了医学图像对比学习中的的挑战，并在多个下游任务中展现出了出色的表现。