The recent vision transformer(i.e.for image classification) learns non-local
attentive interaction of different patch tokens. However, prior arts miss
learning the cross-scale dependencies of different pixels, the semantic
correspondence of different labels, and the consistency of the feature
representations and semantic embeddings, which are critical for biomedical
segmentation. In this paper, we tackle the above issues by proposing a unified
transformer network, termed Multi-Compound Transformer (MCTrans), which
incorporates rich feature learning and semantic structure mining into a unified
framework. Specifically, MCTrans embeds the multi-scale convolutional features
as a sequence of tokens and performs intra- and inter-scale self-attention,
rather than single-scale attention in previous works. In addition, a learnable
proxy embedding is also introduced to model semantic relationship and feature
enhancement by using self-attention and cross-attention, respectively. MCTrans
can be easily plugged into a UNet-like network and attains a significant
improvement over the state-of-the-art methods in biomedical image segmentation
in six standard benchmarks. For example, MCTrans outperforms UNet by 3.64%,
3.71%, 4.34%, 2.8%, 1.88%, 1.57% in Pannuke, CVC-Clinic, CVC-Colon, Etis,
Kavirs, ISIC2018 dataset, respectively. Code is available at
this https URL

本研究提出了一种统一的变形器网络，称为多化合物变形器（MCTrans），可嵌入 UNet-like 网络中，并在生物医学图像分割中显着提高了最先进方法的性能。它使用自我关注和交叉注意力进行多尺度卷积特征的序列嵌入，引入了可学习的代理嵌入来模拟语义关系和特征增强。