Learning continually from a stream of non-i.i.d. data is an open challenge in deep learning, even more so when working in resource-constrained environments such as embedded devices. Visual models that are continually updated through supervised learning are often prone to overfitting, catastrophic forgetting, and biased representations. On the other hand, large language models contain knowledge about multiple concepts and their relations, which can foster a more robust, informed and coherent learning process. This work proposes Continual Visual Mapping (CVM), an approach that continually ground vision representations to a knowledge space extracted from a fixed Language model. Specifically, CVM continually trains a small and efficient visual model to map its representations into a conceptual space established by a fixed Large Language Model. Due to their smaller nature, CVM can be used when directly adapting large visual pre-trained models is unfeasible due to computational or data constraints. CVM overcome state-of-the-art continual learning methods on five benchmarks and offers a promising avenue for addressing generalization capabilities in continual learning, even in computationally constrained devices.

该论文提出了连续视觉映射（Continual Visual Mapping，CVM）的方法，该方法通过从固定的大型语言模型中抽取的知识空间不断将视觉表示与概念空间进行关联，以解决深度学习中不断地从非独立同分布的数据流中学习的问题。CVM在五个基准测试中超越了现有的连续学习方法，为解决计算能力有限的设备上的泛化能力提供了一个有前途的途径。

在资源受限环境中持续学习将视觉概念映射到大型语言模型