In a conventional supervised learning setting, a machine learning model has
access to examples of all object classes that are desired to be recognized
during the inference stage. This results in a fixed model that lacks the
flexibility to adapt to new learning tasks. In practical settings, learning
tasks often arrive in a sequence and the models must continually learn to
increment their previously acquired knowledge. Existing incremental learning
approaches fall well below the state-of-the-art cumulative models that use all
training classes at once. In this paper, we propose a random path selection
algorithm, called Adaptive RPS-Net, that progressively chooses optimal paths
for the new tasks while encouraging parameter sharing between tasks. We
introduce a new network capacity measure that enables us to automatically
switch paths if the already used resources are saturated. Since the proposed
path-reuse strategy ensures forward knowledge transfer, our approach is
efficient and has considerably less computation overhead. As an added novelty,
the proposed model integrates knowledge distillation and retrospection along
with the path selection strategy to overcome catastrophic forgetting. In order
to maintain an equilibrium between previous and newly acquired knowledge, we
propose a simple controller to dynamically balance the model plasticity.
Through extensive experiments, we demonstrate that the Adaptive RPS-Net method
surpasses the state-of-the-art performance for incremental learning and by
utilizing parallel computation this method can run in constant time with nearly
the same efficiency as a conventional deep convolutional neural network.

本文提出了自适应随机路径选择网络（Adaptive RPS-Net）算法，并通过使用新任务的最优路径和任务之间参数共享的方式，提高了网络的灵活性和运行效率。使用知识蒸馏和回顾策略，以及路径重用策略来防止灾难性遗忘，通过动态平衡模型可塑性的简单控制器来维护先前和新获取的知识之间的平衡关系。实验证明，该方法在连续学习和速度方面都具有优势。