Transfer learning from supervised ImageNet models has been frequently used in
medical image analysis. Yet, no large-scale evaluation has been conducted to
benchmark the efficacy of newly-developed pre-training techniques for medical
image analysis, leaving several important questions unanswered. As the first
step in this direction, we conduct a systematic study on the transferability of
models pre-trained on iNat2021, the most recent large-scale fine-grained
dataset, and 14 top self-supervised ImageNet models on 7 diverse medical tasks
in comparison with the supervised ImageNet model. Furthermore, we present a
practical approach to bridge the domain gap between natural and medical images
by continually (pre-)training supervised ImageNet models on medical images. Our
comprehensive evaluation yields new insights: (1) pre-trained models on
fine-grained data yield distinctive local representations that are more
suitable for medical segmentation tasks, (2) self-supervised ImageNet models
learn holistic features more effectively than supervised ImageNet models, and
(3) continual pre-training can bridge the domain gap between natural and
medical images. We hope that this large-scale open evaluation of transfer
learning can direct the future research of deep learning for medical imaging.
As open science, all codes and pre-trained models are available on our GitHub
page this https URL

通过对比自然图像与医学图像，本论文通过大规模评估了不同图像预训练模型在医学图像分析领域的可迁移性，提出了将自然图像模型迁移到医学图像中的实用方法，并发现新的关键性能：针对医学分割任务 fine-grained 数据预训练模型具有独特的局部特征表示，自我监督的 ImageNet 模型比有监督的 ImageNet 更有效地学习整体特征，连续预训练可以缩小自然图像与医学图像领域之间的跨度，有望引导深度学习在医学图像领域的未来研究。