The recent surge of foundation models in computer vision and natural language
processing opens up perspectives in utilizing multi-modal clinical data to
train large models with strong generalizability. Yet pathological image
datasets often lack biomedical text annotation and enrichment. Guiding
data-efficient image diagnosis from the use of biomedical text knowledge
becomes a substantial interest. In this paper, we propose to Connect Image and
Text Embeddings (CITE) to enhance pathological image classification. CITE
injects text insights gained from language models pre-trained with a broad
range of biomedical texts, leading to adapt foundation models towards
pathological image understanding. Through extensive experiments on the
PatchGastric stomach tumor pathological image dataset, we demonstrate that CITE
achieves leading performance compared with various baselines especially when
training data is scarce. CITE offers insights into leveraging in-domain text
knowledge to reinforce data-efficient pathological image classification. Code
is available at this https URL

利用联接图像和文本嵌入（CITE）方法改进病理图像分类，通过注入来自在广泛的生物医学文本中预训练的语言模型所获取的文本见解，使基础模型适应病理图像理解，并在 PatchGastric 胃肿瘤病理图像数据集上取得了领先的性能。