Personalizing a large-scale pretrained Text-to-Image (T2I) diffusion model is
challenging as it typically struggles to make an appropriate trade-off between
its training data distribution and the target distribution, i.e., learning a
novel concept with only a few target images to achieve personalization
(aligning with the personalized target) while preserving text editability
(aligning with diverse text prompts). In this paper, we propose PaRa, an
effective and efficient Parameter Rank Reduction approach for T2I model
personalization by explicitly controlling the rank of the diffusion model
parameters to restrict its initial diverse generation space into a small and
well-balanced target space. Our design is motivated by the fact that taming a
T2I model toward a novel concept such as a specific art style implies a small
generation space. To this end, by reducing the rank of model parameters during
finetuning, we can effectively constrain the space of the denoising sampling
trajectories towards the target. With comprehensive experiments, we show that
PaRa achieves great advantages over existing finetuning approaches on
single/multi-subject generation as well as single-image editing. Notably,
compared to the prevailing fine-tuning technique LoRA, PaRa achieves better
parameter efficiency (2x fewer learnable parameters) and much better target
image alignment.

PaRa 是一种用于 T2I 模型个性化的参数等级减少方法，通过显式控制扩散模型参数的等级来限制其初始的多样化生成空间为一个小而平衡的目标空间，通过全面实验证明，PaRa 在单 / 多主题生成以及单图像编辑方面相比现有的微调方法具有更好的参数效率（2 倍更少的可学习参数）和更好的目标图像对齐效果。