Deep neural networks have demonstrated remarkable performance in medical
image analysis. However, its susceptibility to spurious correlations due to
shortcut learning raises concerns about network interpretability and
reliability. Furthermore, shortcut learning is exacerbated in medical contexts
where disease indicators are often subtle and sparse. In this paper, we propose
a novel gaze-directed Vision GNN (called GD-ViG) to leverage the visual
patterns of radiologists from gaze as expert knowledge, directing the network
toward disease-relevant regions, and thereby mitigating shortcut learning.
GD-ViG consists of a gaze map generator (GMG) and a gaze-directed classifier
(GDC). Combining the global modelling ability of GNNs with the locality of
CNNs, GMG generates the gaze map based on radiologists' visual patterns.
Notably, it eliminates the need for real gaze data during inference, enhancing
the network's practical applicability. Utilizing gaze as the expert knowledge,
the GDC directs the construction of graph structures by incorporating both
feature distances and gaze distances, enabling the network to focus on
disease-relevant foregrounds. Thereby avoiding shortcut learning and improving
the network's interpretability. The experiments on two public medical image
datasets demonstrate that GD-ViG outperforms the state-of-the-art methods, and
effectively mitigates shortcut learning. Our code is available at
this https URL

本研究提出了一种新颖的注视导向视觉图神经网络（称为 GD-ViG），通过利用注视的视觉模式将网络引导到与疾病相关的区域，并从而减轻了快捷学习。该网络在医学图像分析中取得了优异的表现，比最新的方法更有效地减少了快捷学习问题。