Contrastive pretraining provides robust representations by ensuring their
invariance to different image transformations while simultaneously preventing
representational collapse. Equivariant contrastive learning, on the other hand,
provides representations sensitive to specific image transformations while
remaining invariant to others. By introducing equivariance to time-induced
transformations, such as disease-related anatomical changes in longitudinal
imaging, the model can effectively capture such changes in the representation
space. In this work, we pro-pose a Time-equivariant Contrastive Learning (TC)
method. First, an encoder embeds two unlabeled scans from different time points
of the same patient into the representation space. Next, a temporal
equivariance module is trained to predict the representation of a later visit
based on the representation from one of the previous visits and the
corresponding time interval with a novel regularization loss term while
preserving the invariance property to irrelevant image transformations. On a
large longitudinal dataset, our model clearly outperforms existing equivariant
contrastive methods in predicting progression from intermediate age-related
macular degeneration (AMD) to advanced wet-AMD within a specified time-window.

我们提出了一种时间等变对比学习方法，通过引入时间诱发的转换来捕捉纵向成像过程中的变化，该方法在大规模纵向数据集上优于现有的等变对比方法，用于预测中间年龄相关性黄斑变性（AMD）转为先进型湿型 AMD 的进展。

视网膜 OCT 中退行性疾病进展的时间等变对比学习

Time-Equivariant Contrastive Learning for Degenerative Disease  Progression in Retinal OCT

We propose DiffCSE, an unsupervised contrastive learning framework for
learning sentence embeddings. DiffCSE learns sentence embeddings that are
sensitive to the difference between the original sentence and an edited
sentence, where the edited sentence is obtained by stochastically masking out
the original sentence and then sampling from a masked language model. We show
that DiffSCE is an instance of equivariant contrastive learning (Dangovski et
al., 2021), which generalizes contrastive learning and learns representations
that are insensitive to certain types of augmentations and sensitive to other
"harmful" types of augmentations. Our experiments show that DiffCSE achieves
state-of-the-art results among unsupervised sentence representation learning
methods, outperforming unsupervised SimCSE by 2.3 absolute points on semantic
textual similarity tasks.

本文介绍了 DiffCSE，一个无监督的对比学习框架，可以学习敏感于原始句子和编辑后句子之间差异的句子嵌入，通过朴素的掩码和掩码语言模型中的采样获得编辑后的句子，证明了 DiffSCE 是一种等变对比学习，并在语义文本相似性任务中取得了超越无监督 SimCSE 的最先进结果。