Continual learning aims to learn from a stream of continuously arriving data
with minimum forgetting of previously learned knowledge. While previous works
have explored the effectiveness of leveraging the generalizable knowledge from
pre-trained models in continual learning, existing parameter-efficient
fine-tuning approaches focus on the use of a predetermined or task-wise set of
adapters or prompts. However, these approaches still suffer from forgetting due
to task interference on jointly used parameters or restricted flexibility. The
reliance on a static model architecture may lead to the allocation of excessive
parameters that are not essential or, conversely, inadequate adaptation for
downstream tasks, given that the scale and distribution of incoming data are
unpredictable in continual learning. We propose Self-Expansion of pre-trained
models with Modularized Adaptation (SEMA), a novel fine-tuning approach which
automatically decides to reuse or add adapter modules on demand in continual
learning, depending on whether drastic distribution shift that could not be
handled by existing modules is detected at different representation levels. We
design each adapter module to consist of an adapter and a representation
descriptor, specifically, implemented as an autoencoder. The representation
descriptor functions as a distributional shift indicator during training and
triggers adapter expansion. For better usage of the adapters, an expandable
weighting router is learned jointly for mixture of adapter outputs. By
comparing with vision-transformer-based continual learning adaptation methods,
we demonstrate that the proposed framework outperforms the state-of-the-art
without memory rehearsal.

自我扩展且具模块化适应的预训练模型的细调方法 (SEMA) 在连续学习中自动决定何时重用或添加适配器模块，并根据不同表示层检测到的剧烈分布变化来进行适应，通过比较基于视觉转换的连续学习适应方法，证明了该框架优于当前最先进的方法且无需记忆回放。

基于适配器混合的预训练模型自我扩展的持续学习

Self-Expansion of Pre-trained Models with Mixture of Adapters for  Continual Learning

Exponential Moving Average (EMA) is a widely used weight averaging (WA)
regularization to learn flat optima for better generalizations without extra
cost in deep neural network (DNN) optimization. Despite achieving better
flatness, existing WA methods might fall into worse final performances or
require extra test-time computations. This work unveils the full potential of
EMA with a single line of modification, i.e., switching the EMA parameters to
the original model after each epoch, dubbed as Switch EMA (SEMA). From both
theoretical and empirical aspects, we demonstrate that SEMA can help DNNs to
reach generalization optima that better trade-off between flatness and
sharpness. To verify the effectiveness of SEMA, we conduct comparison
experiments with discriminative, generative, and regression tasks on vision and
language datasets, including image classification, self-supervised learning,
object detection and segmentation, image generation, video prediction,
attribute regression, and language modeling. Comprehensive results with popular
optimizers and networks show that SEMA is a free lunch for DNN training by
improving performances and boosting convergence speeds.

利用一行修改将 EMA 参数切换到原始模型并揭示了 Switch EMA（SEMA）的充分潜力，该方法能够帮助深度神经网络在平坦性和尖锐性之间取得更好的泛化最优解，并通过基于视觉和语言数据集的比较实验证实了 SEMA 的有效性。