We propose and study a realistic Continual Learning (CL) setting where
learning algorithms are granted a restricted computational budget per time step
while training. We apply this setting to large-scale semi-supervised Continual
Learning scenarios with sparse label rates. Previous proficient CL methods
perform very poorly in this challenging setting. Overfitting to the sparse
labeled data and insufficient computational budget are the two main culprits
for such a poor performance. Our new setting encourages learning methods to
effectively and efficiently utilize the unlabeled data during training. To that
end, we propose a simple but highly effective baseline, DietCL, which utilizes
both unlabeled and labeled data jointly. DietCL meticulously allocates
computational budget for both types of data. We validate our baseline, at
scale, on several datasets, e.g., CLOC, ImageNet10K, and CGLM, under constraint
budget setups. DietCL outperforms, by a large margin, all existing supervised
CL algorithms as well as more recent continual semi-supervised methods. Our
extensive analysis and ablations demonstrate that DietCL is stable under a full
spectrum of label sparsity, computational budget, and various other ablations.

我们提出并研究了一种现实的连续学习（CL）设置，其中学习算法在训练过程中具有每个时间步的限制计算预算。我们将此设置应用于稀疏标签率的大规模半监督连续学习场景中。我们提出了一种简单但非常有效的基准方法 DietCL，该方法联合利用未标记数据和标记数据，巧妙地分配计算预算。在多个数据集上进行验证时，DietCL 在限制预算的情况下明显优于所有现有的有监督 CL 算法以及更近期的连续半监督方法。我们的广泛分析和消融实验证明，DietCL 在标签稀疏性、计算预算和其他各种消融条件下都是稳定的。