We propose and study a realistic Continual Learning (CL) setting where learning algorithms are granted a restricted computational budget per time step while training. We apply this setting to large-scale semi-supervised Continual Learning scenarios with sparse label rates. Previous proficient CL methods perform very poorly in this challenging setting. Overfitting to the sparse labeled data and insufficient computational budget are the two main culprits for such a poor performance. Our new setting encourages learning methods to effectively and efficiently utilize the unlabeled data during training. To that end, we propose a simple but highly effective baseline, DietCL, which utilizes both unlabeled and labeled data jointly. DietCL meticulously allocates computational budget for both types of data. We validate our baseline, at scale, on several datasets, e.g., CLOC, ImageNet10K, and CGLM, under constraint budget setups. DietCL outperforms, by a large margin, all existing supervised CL algorithms as well as more recent continual semi-supervised methods. Our extensive analysis and ablations demonstrate that DietCL is stable under a full spectrum of label sparsity, computational budget, and various other ablations.

我们提出并研究了一种现实的连续学习（CL）设置，其中学习算法在训练过程中具有每个时间步的限制计算预算。我们将此设置应用于稀疏标签率的大规模半监督连续学习场景中。我们提出了一种简单但非常有效的基准方法DietCL，该方法联合利用未标记数据和标记数据，巧妙地分配计算预算。在多个数据集上进行验证时，DietCL在限制预算的情况下明显优于所有现有的有监督CL算法以及更近期的连续半监督方法。我们的广泛分析和消融实验证明，DietCL在标签稀疏性、计算预算和其他各种消融条件下都是稳定的。

在有限计算资源下学习来自稀疏标注数据流的连续学习