In oncology research, accurate 3D segmentation of lesions from CT scans is
essential for the modeling of lesion growth kinetics. However, following the
RECIST criteria, radiologists routinely only delineate each lesion on the axial
slice showing the largest transverse area, and delineate a small number of
lesions in 3D for research purposes. As a result, we have plenty of unlabeled
3D volumes and labeled 2D images, and scarce labeled 3D volumes, which makes
training a deep-learning 3D segmentation model a challenging task. In this
work, we propose a novel model, denoted a multi-dimension unified Swin
transformer (MDU-ST), for 3D lesion segmentation. The MDU-ST consists of a
Shifted-window transformer (Swin-transformer) encoder and a convolutional
neural network (CNN) decoder, allowing it to adapt to 2D and 3D inputs and
learn the corresponding semantic information in the same encoder. Based on this
model, we introduce a three-stage framework: 1) leveraging large amount of
unlabeled 3D lesion volumes through self-supervised pretext tasks to learn the
underlying pattern of lesion anatomy in the Swin-transformer encoder; 2)
fine-tune the Swin-transformer encoder to perform 2D lesion segmentation with
2D RECIST slices to learn slice-level segmentation information; 3) further
fine-tune the Swin-transformer encoder to perform 3D lesion segmentation with
labeled 3D volumes. The network's performance is evaluated by the Dice
similarity coefficient (DSC) and Hausdorff distance (HD) using an internal 3D
lesion dataset with 593 lesions extracted from multiple anatomical locations.
The proposed MDU-ST demonstrates significant improvement over the competing
models. The proposed method can be used to conduct automated 3D lesion
segmentation to assist radiomics and tumor growth modeling studies. This paper
has been accepted by the IEEE International Symposium on Biomedical Imaging
(ISBI) 2023.

在肿瘤学研究中，准确的 CT 扫描病灶 3D 分割对于病灶生长动力学的建模至关重要。然而，根据 RECIST 准则，放射科医生通常仅在显示最大横截面面积的轴位切片上勾画每个病灶，并在研究目的上勾画少量的 3D 病灶。因此，我们有大量未标记的 3D 体积和带标签的 2D 图像，以及稀缺的标记的 3D 体积，这使得训练深度学习 3D 分割模型成为一项具有挑战性的任务。本研究提出了一种新模型，称为多维统一的 Swin Transformer (MDU-ST)，用于 3D 病灶分割。MDU-ST 由一个偏移窗口变换器 (Swin-transformer) 编码器和一个卷积神经网络 (CNN) 解码器组成，使其能够适应 2D 和 3D 输入，并在同一编码器中学习相应的语义信息。基于该模型，我们引入了一个三阶段框架：1) 通过自我监督的先验任务利用大量未标记的 3D 病灶体积来学习 Swin-transformer 编码器中病灶解剖学的潜在模式；2) 对 Swin-transformer 编码器进行微调，以使用 2D RECIST 切片执行 2D 病灶分割，学习切片级分割信息；3) 进一步对 Swin-transformer 编码器进行微调，以使用带标签的 3D 体积执行 3D 病灶分割。该网络的性能通过 Dice 相似系数 (DSC) 和 Hausdorff 距离 (HD) 在一个内部的 3D 病灶数据集上进行评估，其中包含来自多个解剖位置的 593 个病灶。所提出的 MDU-ST 相比竞争模型表现出显著改进。该方法可用于进行自动化的 3D 病灶分割以辅助放射组学和肿瘤生长建模研究。本论文已被 IEEE 国际生物医学成像研讨会 (ISBI) 2023 接受。