In this paper, we consider the problem of visual representation learning for
computational pathology, by exploiting large-scale image-text pairs gathered
from public resources, along with the domain specific knowledge in pathology.
Specifically, we make the following contributions: (i) We curate a pathology
knowledge tree that consists of 50,470 informative attributes for 4,718
diseases requiring pathology diagnosis from 32 human tissues. To our knowledge,
this is the first comprehensive structured pathology knowledge base; (ii) We
develop a knowledge-enhanced visual-language pretraining approach, where we
first project pathology-specific knowledge into latent embedding space via
language model, and use it to guide the visual representation learning; (iii)
We conduct thorough experiments to validate the effectiveness of our proposed
components, demonstrating significant performance improvement on various
downstream tasks, including cross-modal retrieval, zero-shot classification on
pathology patches, and zero-shot tumor subtyping on whole slide images (WSIs).
All codes, models and the pathology knowledge tree will be released to the
research community

本文研究了计算病理学的视觉表征学习问题，通过利用来自公共资源的大规模图像 - 文本对和病理学中的领域特定知识。我们首次构建了包含 50,470 个有信息量的属性、涵盖 32 种人体组织下的 4,718 种需要病理诊断的疾病的病理知识树。我们开发了一种基于知识增强的视觉 - 语言预训练方法，通过语言模型将病理特定知识投射到潜在嵌入空间中，并用于引导视觉表征学习。我们进行了全面的实验证明了我们提出的组件的有效性，在不同的下游任务中，包括跨模态检索、病理图块上的零样本分类以及整个切片图像上的零样本肿瘤亚型划分，都取得了显著的性能提高。所有代码、模型和病理知识树将提供给研究社群。