Self-supervised representation learning has been highly promising for
histopathology image analysis with numerous approaches leveraging their
patient-slide-patch hierarchy to learn better representations. In this paper,
we explore how the combination of domain specific natural language information
with such hierarchical visual representations can benefit rich representation
learning for medical image tasks. Building on automated language description
generation for features visible in histopathology images, we present a novel
language-tied self-supervised learning framework, Hierarchical Language-tied
Self-Supervision (HLSS) for histopathology images. We explore contrastive
objectives and granular language description based text alignment at multiple
hierarchies to inject language modality information into the visual
representations. Our resulting model achieves state-of-the-art performance on
two medical imaging benchmarks, OpenSRH and TCGA datasets. Our framework also
provides better interpretability with our language aligned representation
space. Code is available at this https URL

本文介绍一种新颖的基于语言相关的自监督学习框架，即 Hierarchical Language-tied Self-Supervised (HLSS) 方法，用于医学图像的层次化表示学习。通过结合领域特定的自然语言信息与图像的层次化视觉表示，该方法在医学图像任务中取得了最先进的性能，同时还提供了更好的可解释性。