Recent personalization methods for diffusion models, such as Dreambooth, allow fine-tuning pre-trained models to generate new concepts. However, applying these techniques across multiple tasks in order to include, e.g., several new objects or styles, leads to mutual interference between their adapters. While recent studies attempt to mitigate this issue by combining trained adapters across tasks after fine-tuning, we adopt a more rigorous regime and investigate the personalization of large diffusion models under a continual learning scenario, where such interference leads to catastrophic forgetting of previous knowledge. To that end, we evaluate the na\"ive continual fine-tuning of customized models and compare this approach with three methods for consecutive adapters' training: sequentially merging new adapters, merging orthogonally initialized adapters, and updating only relevant parameters according to the task. In our experiments, we show that the proposed approaches mitigate forgetting when compared to the na\"ive approach.

本研究解决了现有扩散模型个性化方法在多任务下导致适配器间相互干扰的问题。论文提出了在持续学习场景下对大型扩散模型进行个性化的新方法，通过对比不同的适配器训练方法，发现新的训练策略能够有效减轻知识遗忘。在实验中，所提出的方法表现优于传统的简单持续微调方法。

低秩持续个性化扩散模型