Recent advancements in large-scale Vision Transformers have made significant
strides in improving pre-trained models for medical image segmentation.
However, these methods face a notable challenge in acquiring a substantial
amount of pre-training data, particularly within the medical field. To address
this limitation, we present Masked Multi-view with Swin Transformers (SwinMM),
a novel multi-view pipeline for enabling accurate and data-efficient
self-supervised medical image analysis. Our strategy harnesses the potential of
multi-view information by incorporating two principal components. In the
pre-training phase, we deploy a masked multi-view encoder devised to
concurrently train masked multi-view observations through a range of diverse
proxy tasks. These tasks span image reconstruction, rotation, contrastive
learning, and a novel task that employs a mutual learning paradigm. This new
task capitalizes on the consistency between predictions from various
perspectives, enabling the extraction of hidden multi-view information from 3D
medical data. In the fine-tuning stage, a cross-view decoder is developed to
aggregate the multi-view information through a cross-attention block. Compared
with the previous state-of-the-art self-supervised learning method Swin UNETR,
SwinMM demonstrates a notable advantage on several medical image segmentation
tasks. It allows for a smooth integration of multi-view information,
significantly boosting both the accuracy and data-efficiency of the model. Code
and models are available at this https URL

通过引入多视角信息和自监督学习，本研究提出了一种名为 SwinMM 的新型多视角流水线，以提高医学图像分割预训练模型的准确性和数据效率。