Recent advancements in contrastive learning have revolutionized
self-supervised representation learning and achieved state-of-the-art
performance on benchmark tasks. While most existing methods focus on applying
contrastive learning to input data modalities such as images, natural language
sentences, or networks, they overlook the potential of utilizing outputs from
previously trained encoders. In this paper, we introduce SIMSKIP, a novel
contrastive learning framework that specifically refines input embeddings for
downstream tasks. Unlike traditional unsupervised learning approaches, SIMSKIP
takes advantage of the output embeddings of encoder models as its input.
Through theoretical analysis, we provide evidence that applying SIMSKIP does
not result in larger upper bounds on downstream task errors than those of the
original embeddings, which serve as SIMSKIP's input. Experimental results on
various open datasets demonstrate that the embeddings produced by SIMSKIP
improve performance on downstream tasks.

使用 SIMSKIP，一种新颖的对比学习框架，通过改善输入嵌入来提高后续任务的性能。SIMSKIP 利用之前训练过的编码器模型的输出嵌入作为输入，通过理论分析证明其不会导致下游任务错误的上限增加。在不同的开放数据集上的实验证明 SIMSKIP 产生的嵌入能够提升后续任务的性能。