Class incremental learning (CIL) is a challenging setting of continual
learning, which learns a series of tasks sequentially. Each task consists of a
set of unique classes. The key feature of CIL is that no task identifier (or
task-id) is provided at test time for each test sample. Predicting the task-id
for each test sample is a challenging problem. An emerging theoretically
justified and effective approach is to train a task-specific model for each
task in a shared network for all tasks based on a task-incremental learning
(TIL) method to deal with forgetting. The model for each task in this approach
is an out-of-distribution (OOD) detector rather than a conventional classifier.
The OOD detector can perform both within-task (in-distribution (IND)) class
prediction and OOD detection. The OOD detection capability is the key for
task-id prediction during inference for each test sample. However, this paper
argues that using a traditional OOD detector for task-id prediction is
sub-optimal because additional information (e.g., the replay data and the
learned tasks) available in CIL can be exploited to design a better and
principled method for task-id prediction. We call the new method TPLR (Task-id
Prediction based on Likelihood Ratio}). TPLR markedly outperforms strong CIL
baselines.

在连续学习中，对于每个测试样本预测任务标识符是一个具有挑战性的问题。本文提出了一种新的方法 TPLR（基于似然比的任务标识符预测），该方法利用了连续学习中的额外信息，显著优于强对比模型。

基于似然比的任务预测的类增量学习

Class Incremental Learning via Likelihood Ratio Based Task Prediction

Multi-task learning (MTL) is to learn one single model that performs multiple
tasks for achieving good performance on all tasks and lower cost on
computation. Learning such a model requires to jointly optimize losses of a set
of tasks with different difficulty levels, magnitudes, and characteristics
(e.g. cross-entropy, Euclidean loss), leading to the imbalance problem in
multi-task learning. To address the imbalance problem, we propose a knowledge
distillation based method in this work. We first learn a task-specific model
for each task. We then learn the multi-task model for minimizing task-specific
loss and for producing the same feature with task-specific models. As the
task-specific network encodes different features, we introduce small
task-specific adaptors to project multi-task features to the task-specific
features. In this way, the adaptors align the task-specific feature and the
multi-task feature, which enables a balanced parameter sharing across tasks.
Extensive experimental results demonstrate that our method can optimize a
multi-task learning model in a more balanced way and achieve better overall
performance.

本文提出了一种基于知识蒸馏的多任务学习方法。通过先学习任务特定的模型，再学习多任务模型，该方法可以解决多任务学习中的不平衡问题，以实现更好的整体性能。