Pre-training strategies based on self-supervised learning (SSL) have proven
to be effective pretext tasks for many downstream tasks in computer vision. Due
to the significant disparity between medical and natural images, the
application of typical SSL is not straightforward in medical imaging.
Additionally, those pretext tasks often lack context, which is critical for
computer-aided clinical decision support. In this paper, we developed a
longitudinal masked auto-encoder (MAE) based on the well-known
Transformer-based MAE. In particular, we explored the importance of time-aware
position embedding as well as disease progression-aware masking. Taking into
account the time between examinations instead of just scheduling them offers
the benefit of capturing temporal changes and trends. The masking strategy, for
its part, evolves during follow-up to better capture pathological changes,
ensuring a more accurate assessment of disease progression. Using OPHDIAT, a
large follow-up screening dataset targeting diabetic retinopathy (DR), we
evaluated the pre-trained weights on a longitudinal task, which is to predict
the severity label of the next visit within 3 years based on the past time
series examinations. Our results demonstrated the relevancy of both time-aware
position embedding and masking strategies based on disease progression
knowledge. Compared to popular baseline models and standard longitudinal
Transformers, these simple yet effective extensions significantly enhance the
predictive ability of deep classification models.

本文提出基于自监督学习（SSL）的预训练策略，针对医学图像的非典型性，开发了一种基于 Transformer 的纵向遮蔽自编码器 (MAE)，研究了时间感知的位置嵌入和疾病进展感知的遮蔽策略，通过评估在糖尿病视网膜病变（DR）筛查数据集上的预训练权重，证明了这些扩展显著提高深度分类模型的预测能力。