This work explores the effectiveness of masked image modelling for learning
representations of retinal OCT images. To this end, we leverage Masked
Autoencoders (MAE), a simple and scalable method for self-supervised learning,
to obtain a powerful and general representation for OCT images by training on
700K OCT images from 41K patients collected under real world clinical settings.
We also provide the first extensive evaluation for a model of OCT on a
challenging battery of 6 downstream tasks. Our model achieves strong
performance when fully finetuned but can also serve as a versatile frozen
feature extractor for many tasks using lightweight adapters. Furthermore, we
propose an extension of the MAE pretraining to fuse OCT with an auxiliary
modality, namely, IR fundus images and learn a joint model for both. We
demonstrate our approach improves performance on a multimodal downstream
application. Our experiments utilize most publicly available OCT datasets, thus
enabling future comparisons. Our code and model weights are publicly available
this https URL

本研究探讨了使用遮罩图像建模来学习视网膜 OCT 图像的表示的有效性。我们利用遮罩自编码器（MAE）这一简单且可扩展的自监督学习方法，在真实世界的临床环境中使用来自 41K 患者的 700K OCT 图像进行训练，得出了有力且通用的 OCT 图像表示。我们还对 OCT 模型进行了首次广泛评估，通过六个具有挑战性的下游任务，完全微调后的模型表现出了良好的性能，并可以作为轻量级适配器的多功能冻结特征提取器。此外，我们提出了一种扩展的 MAE 预训练方法，将 OCT 与辅助模态 —— 即红外眼底图像相结合，并为两者学习一个联合模型。我们通过多模态下游应用的实验证明了我们的方法提高了性能。我们的实验利用了大多数公开可用的 OCT 数据集，从而为未来的比较工作提供了可能。我们的代码和模型权重公开可用于此 https URL。