Computer-aided medical image segmentation has been applied widely in
diagnosis and treatment to obtain clinically useful information of shapes and
volumes of target organs and tissues. In the past several years, convolutional
neural network (CNN) based methods (e.g., U-Net) have dominated this area, but
still suffered from inadequate long-range information capturing. Hence, recent
work presented computer vision Transformer variants for medical image
segmentation tasks and obtained promising performances. Such Transformers model
long-range dependency by computing pair-wise patch relations. However, they
incur prohibitive computational costs, especially on 3D medical images (e.g.,
CT and MRI). In this paper, we propose a new method called Dilated Transformer,
which conducts self-attention for pair-wise patch relations captured
alternately in local and global scopes. Inspired by dilated convolution
kernels, we conduct the global self-attention in a dilated manner, enlarging
receptive fields without increasing the patches involved and thus reducing
computational costs. Based on this design of Dilated Transformer, we construct
a U-shaped encoder-decoder hierarchical architecture called D-Former for 3D
medical image segmentation. Experiments on the Synapse and ACDC datasets show
that our D-Former model, trained from scratch, outperforms various competitive
CNN-based or Transformer-based segmentation models at a low computational cost
without time-consuming per-training process.

本文提出了一种称为 Dilated Transformer 的方法，该方法在全球和局部范围内交替捕捉成对的补丁关系进行自我关注。我们基于 Dilated Transformer 的设计构建了一种名为 D-Former 的 U 型编码器 - 解码器分层结构，用于 3D 医学图像分割。实验表明，我们的 D-Former 模型在低计算成本下，从头开始训练，优于各种有竞争力的基于 CNN 或 Transformer 的分割模型。