Convolutional neural networks (CNNs) have been the de facto standard for
nowadays 3D medical image segmentation. The convolutional operations used in
these networks, however, inevitably have limitations in modeling the long-range
dependency due to their inductive bias of locality and weight sharing. Although
Transformer was born to address this issue, it suffers from extreme
computational and spatial complexities in processing high-resolution 3D feature
maps. In this paper, we propose a novel framework that efficiently bridges a
{\bf Co}nvolutional neural network and a {\bf Tr}ansformer {\bf (CoTr)} for
accurate 3D medical image segmentation. Under this framework, the CNN is
constructed to extract feature representations and an efficient deformable
Transformer (DeTrans) is built to model the long-range dependency on the
extracted feature maps. Different from the vanilla Transformer which treats all
image positions equally, our DeTrans pays attention only to a small set of key
positions by introducing the deformable self-attention mechanism. Thus, the
computational and spatial complexities of DeTrans have been greatly reduced,
making it possible to process the multi-scale and high-resolution feature maps,
which are usually of paramount importance for image segmentation. We conduct an
extensive evaluation on the Multi-Atlas Labeling Beyond the Cranial Vault (BCV)
dataset that covers 11 major human organs. The results indicate that our CoTr
leads to a substantial performance improvement over other CNN-based,
transformer-based, and hybrid methods on the 3D multi-organ segmentation task.
Code is available at \def\UrlFont{\rm\small\ttfamily}
https://github.com/YtongXie/CoTr

提出了一种 CoTr 框架，将卷积神经网络和 Transformer 相结合，利用 DeTrans 的可变形注意机制有效地处理高分辨率的 3D 医学图像，对 11 个主要人体器官进行的广泛评估显示，CoTr 相对于其他 CNN、Transformer 和混合方法在 3D 多器官分割任务中具有显著的性能优势。