Advanced life forms, sustained by the synergistic interaction of neural
cognitive mechanisms, continually acquire and transfer knowledge throughout
their lifespan. In contrast, contemporary machine learning paradigms exhibit
limitations in emulating the facets of continual learning (CL). Nonetheless,
the emergence of large language models (LLMs) presents promising avenues for
realizing CL via interactions with these models. Drawing on Complementary
Learning System theory, this paper presents a novel Interactive Continual
Learning (ICL) framework, enabled by collaborative interactions among models of
various sizes. Specifically, we assign the ViT model as System1 and multimodal
LLM as System2. To enable the memory module to deduce tasks from class
information and enhance Set2Set retrieval, we propose the Class-Knowledge-Task
Multi-Head Attention (CKT-MHA). Additionally, to improve memory retrieval in
System1 through enhanced geometric representation, we introduce the CL-vMF
mechanism, based on the von Mises-Fisher (vMF) distribution. Meanwhile, we
introduce the von Mises-Fisher Outlier Detection and Interaction (vMF-ODI)
strategy to identify hard examples, thus enhancing collaboration between
System1 and System2 for complex reasoning realization. Comprehensive evaluation
of our proposed ICL demonstrates significant resistance to forgetting and
superior performance relative to existing methods.

基于互动持续学习框架，使用大型语言模型与记忆检索机制，以及模型之间的协作交互，实现抵抗遗忘和优越性能的持续学习。