Recent work has demonstrated a remarkable ability to customize text-to-image diffusion models to multiple, fine-grained concepts in a sequential (i.e., continual) manner while only providing a few example images for each concept. This setting is known as continual diffusion. Here, we ask the question: Can we scale these methods to longer concept sequences without forgetting? Although prior work mitigates the forgetting of previously learned concepts, we show that its capacity to learn new tasks reaches saturation over longer sequences. We address this challenge by introducing a novel method, STack-And-Mask INcremental Adapters (STAMINA), which is composed of low-ranked attention-masked adapters and customized MLP tokens. STAMINA is designed to enhance the robust fine-tuning properties of LoRA for sequential concept learning via learnable hard-attention masks parameterized with low rank MLPs, enabling precise, scalable learning via sparse adaptation. Notably, all introduced trainable parameters can be folded back into the model after training, inducing no additional inference parameter costs. We show that STAMINA outperforms the prior SOTA for the setting of text-to-image continual customization on a 50-concept benchmark composed of landmarks and human faces, with no stored replay data. Additionally, we extended our method to the setting of continual learning for image classification, demonstrating that our gains also translate to state-of-the-art performance in this standard benchmark.

我们引入了一种名为STAMINA的新方法，它由低秩注意蒙版适配器和定制的MLP令牌组成，通过用低秩MLP参数化的可学习硬注意蒙版来增强LoRA在顺序概念学习中的鲁棒微调性能，从而实现精确、可扩展的稀疏适应学习。我们展示了STAMINA在50个概念基准测试中超越了先前最先进的文本到图像连续定制设置，在地标和人脸上没有存储的重播数据。此外，我们还将我们的方法扩展到图像分类的持续学习设置中，证明了我们在这一标准基准测试中的卓越性能转化。

持续扩散与STAMINA: STack-And-Mask 增量适配器