A key challenge for machine intelligence is to learn new visual concepts
without forgetting the previously acquired knowledge. Continual learning is
aimed towards addressing this challenge. However, there is a gap between
existing supervised continual learning and human-like intelligence, where human
is able to learn from both labeled and unlabeled data. How unlabeled data
affects learning and catastrophic forgetting in the continual learning process
remains unknown. To explore these issues, we formulate a new semi-supervised
continual learning method, which can be generically applied to existing
continual learning models. Specifically, a novel gradient learner learns from
labeled data to predict gradients on unlabeled data. Hence, the unlabeled data
could fit into the supervised continual learning method. Different from
conventional semi-supervised settings, we do not hypothesize that the
underlying classes, which are associated to the unlabeled data, are known to
the learning process. In other words, the unlabeled data could be very distinct
from the labeled data. We evaluate the proposed method on mainstream continual
learning, adversarial continual learning, and semi-supervised learning tasks.
The proposed method achieves state-of-the-art performance on classification
accuracy and backward transfer in the continual learning setting while
achieving desired performance on classification accuracy in the semi-supervised
learning setting. This implies that the unlabeled images can enhance the
generalizability of continual learning models on the predictive ability on
unseen data and significantly alleviate catastrophic forgetting. The code is
available at https://github.com/luoyan407/grad_prediction.git.

论文探索了半监督持续学习方法，该方法能从标记数据中预测未标记数据的梯度，进而将未标记数据应用到有监督持续学习中。实验结果表明，未标记图像可以增强持续学习模型对未见数据的预测能力并显著缓解灾难性遗忘。