Transformer architecture has emerged to be successful in a number of natural language processing tasks. However, its applications to medical vision remain largely unexplored. In this study, we present UTNet, a simple yet powerful hybrid Transformer architecture that integrates self-attention into a convolutional neural network for enhancing medical image segmentation. UTNet applies self-attention modules in both encoder and decoder for capturing long-range dependency at different scales with minimal overhead. To this end, we propose an efficient self-attention mechanism along with relative position encoding that reduces the complexity of self-attention operation significantly from $O(n^2)$ to approximate $O(n)$. A new self-attention decoder is also proposed to recover fine-grained details from the skipped connections in the encoder. Our approach addresses the dilemma that Transformer requires huge amounts of data to learn vision inductive bias. Our hybrid layer design allows the initialization of Transformer into convolutional networks without a need of pre-training. We have evaluated UTNet on the multi-label, multi-vendor cardiac magnetic resonance imaging cohort. UTNet demonstrates superior segmentation performance and robustness against the state-of-the-art approaches, holding the promise to generalize well on other medical image segmentations.

本研究提出UTNet，这是一种简单而强大的混合Transformer体系结构，它将self-attention集成到卷积神经网络中，以增强医学图像分割，通过在编码器和解码器中应用self-attention模块来捕捉不同尺度的长程依赖关系，并提出了一种有效的self-attention机制与相对位置编码，从而将self-attention操作的复杂度从 $O(n^2)$ 降低到约 $O(n)$。在多标签、多厂商的心脏磁共振成像队列上对UTNet进行了评估，所示分割性能优异，对抗最先进的方法表现出很好的鲁棒性，有望在其他医学图像分割中具有广泛的应用前景。

UTNet：一种用于医学图像分割的混合Transformer架构