Text-to-image (T2I) personalization allows users to guide the creative image
generation process by combining their own visual concepts in natural language
prompts. Recently, encoder-based techniques have emerged as a new effective
approach for T2I personalization, reducing the need for multiple images and
long training times. However, most existing encoders are limited to a
single-class domain, which hinders their ability to handle diverse concepts. In
this work, we propose a domain-agnostic method that does not require any
specialized dataset or prior information about the personalized concepts. We
introduce a novel contrastive-based regularization technique to maintain high
fidelity to the target concept characteristics while keeping the predicted
embeddings close to editable regions of the latent space, by pushing the
predicted tokens toward their nearest existing CLIP tokens. Our experimental
results demonstrate the effectiveness of our approach and show how the learned
tokens are more semantic than tokens predicted by unregularized models. This
leads to a better representation that achieves state-of-the-art performance
while being more flexible than previous methods.

本文提出了一种基于编码器的文本到图像（T2I）个性化生成方法，使用一种对领域不敏感的对比度正则化技术来实现目标概念特征的高保真度，并将预测的嵌入值保持在可编辑的潜在空间区域，从而实现了基于 CLIP（一种联合学习图像和文本的新型表示方法）标记的最短距离。实验结果证明了我们方法的有效性和学习标记比未规范化模型预测的标记更具语义性的优点，并且比以前的方法更加灵活，表现出最先进的性能。