Pretraining CNN models (i.e., UNet) through self-supervision has become a
powerful approach to facilitate medical image segmentation under low annotation
regimes. Recent contrastive learning methods encourage similar global
representations when the same image undergoes different transformations, or
enforce invariance across different image/patch features that are intrinsically
correlated. However, CNN-extracted global and local features are limited in
capturing long-range spatial dependencies that are essential in biological
anatomy. To this end, we present a keypoint-augmented fusion layer that
extracts representations preserving both short- and long-range self-attention.
In particular, we augment the CNN feature map at multiple scales by
incorporating an additional input that learns long-range spatial self-attention
among localized keypoint features. Further, we introduce both global and local
self-supervised pretraining for the framework. At the global scale, we obtain
global representations from both the bottleneck of the UNet, and by aggregating
multiscale keypoint features. These global features are subsequently
regularized through image-level contrastive objectives. At the local scale, we
define a distance-based criterion to first establish correspondences among
keypoints and encourage similarity between their features. Through extensive
experiments on both MRI and CT segmentation tasks, we demonstrate the
architectural advantages of our proposed method in comparison to both CNN and
Transformer-based UNets, when all architectures are trained with randomly
initialized weights. With our proposed pretraining strategy, our method further
outperforms existing SSL methods by producing more robust self-attention and
achieving state-of-the-art segmentation results. The code is available at
this https URL

通过自我监督预训练的卷积神经网络（即 UNet）已经成为在低标注情况下促进医学图像分割的强大方法。我们提出了一种关键点增强的融合层，它提取保留短程和长程自我注意力的表示，并且在全局和局部尺度上引入了自我监督预训练，通过创建关键点之间的对应关系并鼓励特征之间的相似性来提高分割结果。