Graph-based methods have been extensively applied to whole-slide
histopathology image (WSI) analysis due to the advantage of modeling the
spatial relationships among different entities. However, most of the existing
methods focus on modeling WSIs with homogeneous graphs (e.g., with homogeneous
node type). Despite their successes, these works are incapable of mining the
complex structural relations between biological entities (e.g., the diverse
interaction among different cell types) in the WSI. We propose a novel
heterogeneous graph-based framework to leverage the inter-relationships among
different types of nuclei for WSI analysis. Specifically, we formulate the WSI
as a heterogeneous graph with "nucleus-type" attribute to each node and a
semantic similarity attribute to each edge. We then present a new
heterogeneous-graph edge attribute transformer (HEAT) to take advantage of the
edge and node heterogeneity during massage aggregating. Further, we design a
new pseudo-label-based semantic-consistent pooling mechanism to obtain
graph-level features, which can mitigate the over-parameterization issue of
conventional cluster-based pooling. Additionally, observing the limitations of
existing association-based localization methods, we propose a causal-driven
approach attributing the contribution of each node to improve the
interpretability of our framework. Extensive experiments on three public TCGA
benchmark datasets demonstrate that our framework outperforms the
state-of-the-art methods with considerable margins on various tasks. Our codes
are available at this https URL

我们提出了一种新的基于异构图的框架，利用细胞核的不同类型之间的相互关系进行 WSI 分析。通过将 WSI 建模为一个具有 “核型” 属性和语义相似性属性的异构图，采用新的异构图边属性转换器 (HEAT) 和基于伪标签的语义一致汇聚机制来获得图层特征，并采用因果驱动的方法来解决现有关联定位方法的局限性。大量实验证明，在各种任务上，我们的框架相比于现有方法具有显著的优势。