Identifying anatomical structures (e.g., lesions or landmarks) in medical
images plays a fundamental role in medical image analysis. As an exemplar-based
landmark detection method, Self-supervised Anatomical eMbedding (SAM) learns a
discriminative embedding for each voxel in the image and has shown promising
results on various tasks. However, SAM still faces challenges in: (1)
differentiating voxels with similar appearance but different semantic meanings
(\textit{e.g.}, two adjacent structures without clear borders); (2) matching
voxels with similar semantics but markedly different appearance (e.g., the same
vessel before and after contrast injection); and (3) cross-modality matching
(e.g., CT-MRI registration). To overcome these challenges, we propose SAMv2,
which is a unified framework designed to learn appearance, semantic, and
cross-modality anatomical embeddings. Specifically, SAMv2 incorporates three
key innovations: (1) semantic embedding learning with prototypical contrastive
loss; (2) a fixed-point-based matching strategy; and (3) an iterative approach
for cross-modality embedding learning. We thoroughly evaluated SAMv2 across
three tasks, including one-shot landmark detection, lesion tracking on
longitudinal CT scans, and CT-MRI affine/rigid registration with varying field
of view. Our results suggest that SAMv2 outperforms SAM and other
state-of-the-art methods, offering a robust and versatile approach for landmark
based medical image analysis tasks. Code and trained models are available at:
this https URL

基于自我监督解剖嵌入（SAM）的 SAMv2 是一个统一的框架，通过学习外观、语义和跨模态的解剖嵌入，克服了医学图像分析中的挑战，提供了一种在基于标志点的医学图像分析任务中表现优异的方法。

SAMv2: 统一的外观、语义和跨模态解剖嵌入学习框架

SAMv2: A Unified Framework for Learning Appearance, Semantic and  Cross-Modality Anatomical Embeddings

Radiological images such as computed tomography (CT) and X-rays render
anatomy with intrinsic structures. Being able to reliably locate the same
anatomical structure across varying images is a fundamental task in medical
image analysis. In principle it is possible to use landmark detection or
semantic segmentation for this task, but to work well these require large
numbers of labeled data for each anatomical structure and sub-structure of
interest. A more universal approach would learn the intrinsic structure from
unlabeled images. We introduce such an approach, called Self-supervised
Anatomical eMbedding (SAM). SAM generates semantic embeddings for each image
pixel that describes its anatomical location or body part. To produce such
embeddings, we propose a pixel-level contrastive learning framework. A
coarse-to-fine strategy ensures both global and local anatomical information
are encoded. Negative sample selection strategies are designed to enhance the
embedding's discriminability. Using SAM, one can label any point of interest on
a template image and then locate the same body part in other images by simple
nearest neighbor searching. We demonstrate the effectiveness of SAM in multiple
tasks with 2D and 3D image modalities. On a chest CT dataset with 19 landmarks,
SAM outperforms widely-used registration algorithms while only taking 0.23
seconds for inference. On two X-ray datasets, SAM, with only one labeled
template image, surpasses supervised methods trained on 50 labeled images. We
also apply SAM on whole-body follow-up lesion matching in CT and obtain an
accuracy of 91%. SAM can also be applied for improving image registration and
initializing CNN weights.

Self-supervised Anatomical eMbedding (SAM) is introduced to extract semantic embeddings for imaging pixels with intrinsic structures, which can be used to locate body parts in other images by nearest neighbor searching, surpassing supervised methods trained on 50 labeled images with only one labeled template image and being effective in improving image registration and initializing CNN weights.