Transformers have emerged as the state-of-the-art architecture in medical
image registration, outperforming convolutional neural networks (CNNs) by
addressing their limited receptive fields and overcoming gradient instability
in deeper models. Despite their success, transformer-based models require
substantial resources for training, including data, memory, and computational
power, which may restrict their applicability for end users with limited
resources. In particular, existing transformer-based 3D image registration
architectures face three critical gaps that challenge their efficiency and
effectiveness. Firstly, while mitigating the quadratic complexity of full
attention by focusing on local regions, window-based attention mechanisms often
fail to adequately integrate local and global information. Secondly, feature
similarities across attention heads that were recently found in multi-head
attention architectures indicate a significant computational redundancy,
suggesting that the capacity of the network could be better utilized to enhance
performance. Lastly, the granularity of tokenization, a key factor in
registration accuracy, presents a trade-off; smaller tokens improve detail
capture at the cost of higher computational complexity, increased memory
demands, and a risk of overfitting. Here, we propose EfficientMorph, a
transformer-based architecture for unsupervised 3D image registration. It
optimizes the balance between local and global attention through a plane-based
attention mechanism, reduces computational redundancy via cascaded group
attention, and captures fine details without compromising computational
efficiency, thanks to a Hi-Res tokenization strategy complemented by merging
operations. Notably, EfficientMorph sets a new benchmark for performance on the
OASIS dataset with 16-27x fewer parameters.

EfficientMorph 是一种基于 transformer 的无监督三维图像配准架构，通过平面注意机制优化本地和全局注意的平衡，通过级联组注意减少计算冗余，并通过 Hi-Res 令牌化策略捕捉细节，同时保持计算效率，在 OASIS 数据集上创下了 16-27 倍少参数的性能新纪录。