The emerging scale segmentation model, Segment Anything (SAM), exhibits
impressive capabilities in zero-shot segmentation for natural images. However,
when applied to medical images, SAM suffers from noticeable performance drop.
To make SAM a real ``foundation model" for the computer vision community, it is
critical to find an efficient way to customize SAM for medical image dataset.
In this work, we propose to freeze SAM encoder and finetune a lightweight
task-specific prediction head, as most of weights in SAM are contributed by the
encoder. In addition, SAM is a promptable model, while prompt is not
necessarily available in all application cases, and precise prompts for
multiple class segmentation are also time-consuming. Therefore, we explore
three types of prompt-free prediction heads in this work, include ViT, CNN, and
linear layers. For ViT head, we remove the prompt tokens in the mask decoder of
SAM, which is named AutoSAM. AutoSAM can also generate masks for different
classes with one single inference after modification. To evaluate the
label-efficiency of our finetuning method, we compare the results of these
three prediction heads on a public medical image segmentation dataset with
limited labeled data. Experiments demonstrate that finetuning SAM significantly
improves its performance on medical image dataset, even with just one labeled
volume. Moreover, AutoSAM and CNN prediction head also has better segmentation
accuracy than training from scratch and self-supervised learning approaches
when there is a shortage of annotations.

本文介绍了一种针对医学影像的零样本分割模型 Segment Anything (SAM)，通过细调轻量级任务特定的 ViT 预测头来提高其在医学影像数据集上的性能。

如何高效地将大型分割模型（SAM）适应于医学图像

How to Efficiently Adapt Large Segmentation Model(SAM) to Medical Images

Multi-modal image registration is a challenging problem that is also an
important clinical task for many real applications and scenarios. As a first
step in analysis, deformable registration among different image modalities is
often required in order to provide complementary visual information. During
registration, semantic information is key to match homologous points and
pixels. Nevertheless, many conventional registration methods are incapable in
capturing high-level semantic anatomical dense correspondences. In this work,
we propose a novel multi-task learning system, JSSR, based on an end-to-end 3D
convolutional neural network that is composed of a generator, a registration
and a segmentation component. The system is optimized to satisfy the implicit
constraints between different tasks in an unsupervised manner. It first
synthesizes the source domain images into the target domain, then an
intra-modal registration is applied on the synthesized images and target
images. The segmentation module are then applied on the synthesized and target
images, providing additional cues based on semantic correspondences. The
supervision from another fully-annotated dataset is used to regularize the
segmentation. We extensively evaluate JSSR on a large-scale medical image
dataset containing 1,485 patient CT imaging studies of four different contrast
phases (i.e., 5,940 3D CT scans with pathological livers) on the registration,
segmentation and synthesis tasks. The performance is improved after joint
training on the registration and segmentation tasks by 0.9% and 1.9%
respectively compared to a highly competitive and accurate deep learning
baseline. The registration also consistently outperforms conventional
state-of-the-art multi-modal registration methods.

本文提出了一种基于端到端 3D 卷积神经网络的多任务学习系统 JSSR，用于医学图像的多模态图像配准、分割和合成，该方法能够有效捕获高级语义密集对应，优于市场上已有的相关方法。

JSSR: 三维多模态病理 CT 扫描的联合合成、分割和配准系统

JSSR: A Joint Synthesis, Segmentation, and Registration System for 3D  Multi-Modal Image Alignment of Large-scale Pathological CT Scans

Unsupervised learning on imbalanced data is challenging because, when given
imbalanced data, current model is often dominated by the major category and
ignores the categories with small amount of data. We develop a latent variable
model that can cope with imbalanced data by dividing the latent space into a
shared space and a private space. Based on Gaussian Process Latent Variable
Models, we propose a new kernel formulation that enables the separation of
latent space and derives an efficient variational inference method. The
performance of our model is demonstrated with an imbalanced medical image
dataset.

提出一种分割潜在空间的混合变量模型，基于高斯过程潜在变量模型提出一种新的核函数，并构建出高效的变分推理方法以应对不平衡数据。在不平衡的医学图像数据集上进行实验，证明该模型具有良好的性能。