Continual learning (CL) aims to learn a sequence of tasks without forgetting
the previously acquired knowledge. However, recent CL advances are restricted
to supervised continual learning (SCL) scenarios. Consequently, they are not
scalable to real-world applications where the data distribution is often biased
and unannotated. In this work, we focus on unsupervised continual learning
(UCL), where we learn the feature representations on an unlabelled sequence of
tasks and show that reliance on annotated data is not necessary for continual
learning. We conduct a systematic study analyzing the learned feature
representations and show that unsupervised visual representations are
surprisingly more robust to catastrophic forgetting, consistently achieve
better performance, and generalize better to out-of-distribution tasks than
SCL. Furthermore, we find that UCL achieves a smoother loss landscape through
qualitative analysis of the learned representations and learns meaningful
feature representations. Additionally, we propose Lifelong Unsupervised Mixup
(LUMP), a simple yet effective technique that interpolates between the current
task and previous tasks' instances to alleviate catastrophic forgetting for
unsupervised representations.

该研究提出了一种基于无监督学习方法的连续学习技术，可以在不需要人工标注的数据的情况下设法学习连续任务的特征表示，并通过使用 LUMP 技术来缓解灾难性遗忘问题。