Continual Learning methods are designed to learn new tasks without erasing
previous knowledge. However, Continual Learning often requires massive
computational power and storage capacity for satisfactory performance. In this
paper, we propose a resource-efficient continual learning method called the
Elastic Expansion Network (E2Net). Leveraging core subnet distillation and
precise replay sample selection, E2Net achieves superior average accuracy and
diminished forgetting within the same computational and storage constraints,
all while minimizing processing time. In E2Net, we propose Representative
Network Distillation to identify the representative core subnet by assessing
parameter quantity and output similarity with the working network, distilling
analogous subnets within the working network to mitigate reliance on rehearsal
buffers and facilitating knowledge transfer across previous tasks. To enhance
storage resource utilization, we then propose Subnet Constraint Experience
Replay to optimize rehearsal efficiency through a sample storage strategy based
on the structures of representative networks. Extensive experiments conducted
predominantly on cloud environments with diverse datasets and also spanning the
edge environment demonstrate that E2Net consistently outperforms
state-of-the-art methods. In addition, our method outperforms competitors in
terms of both storage and computational requirements.

我们提出了一种资源有效的连续学习方法，称为弹性扩展网络（E2Net），通过核心子网络精简和精准回放样本选择，实现了卓越的平均准确性和降低遗忘，在相同的计算和存储限制条件下，同时最小化处理时间。