Recent breakthroughs in self-supervised learning show that such algorithms learn visual representations that can be transferred better to unseen tasks than joint-training methods relying on task-specific supervision. In this paper, we found that the similar holds in the continual learning con-text: contrastively learned representations are more robust against the catastrophic forgetting than jointly trained representations. Based on this novel observation, we propose a rehearsal-based continual learning algorithm that focuses on continually learning and maintaining transferable representations. More specifically, the proposed scheme (1) learns representations using the contrastive learning objective, and (2) preserves learned representations using a self-supervised distillation step. We conduct extensive experimental validations under popular benchmark image classification datasets, where our method sets the new state-of-the-art performance.

本文发现在连续学习的情境中，通过对比学习方法学习到的表示对抗遗忘更加鲁棒，基于该观察，我们提出了一种基于重复学习的算法，该算法着重于持续学习和维护可转移的表示，并在常见的基准图像分类数据集上进行了广泛实验验证，该方法取得了最新的最佳性能。

Co$^2$L：对比连续学习