Personalized text-to-image generation has attracted unprecedented attention
in the recent few years due to its unique capability of generating
highly-personalized images via using the input concept dataset and novel
textual prompt. However, previous methods solely focus on the performance of
the reconstruction task, degrading its ability to combine with different
textual prompt. Besides, optimizing in the high-dimensional embedding space
usually leads to unnecessary time-consuming training process and slow
convergence. To address these issues, we propose an efficient method to explore
the target embedding in a textual subspace, drawing inspiration from the
self-expressiveness property. Additionally, we propose an efficient selection
strategy for determining the basis vectors of the textual subspace. The
experimental evaluations demonstrate that the learned embedding can not only
faithfully reconstruct input image, but also significantly improves its
alignment with novel input textual prompt. Furthermore, we observe that
optimizing in the textual subspace leads to an significant improvement of the
robustness to the initial word, relaxing the constraint that requires users to
input the most relevant initial word. Our method opens the door to more
efficient representation learning for personalized text-to-image generation.

我们提出了一种有效的方法来在文本子空间中探索目标嵌入，借鉴了自我表现性特性，并提出了一种有效的选择策略来确定文本子空间的基向量。实验评估表明，所学习的嵌入不仅能够忠实地重构输入图像，还能显著改善其与新的输入文本提示的对齐性。此外，我们观察到在文本子空间中优化能够显著提高对初始词的鲁棒性，从而放宽了用户需要输入最相关初始词的约束。我们的方法为个性化文本到图像生成的更高效的表示学习打开了大门。

通过利用文本子空间提高高效个性化文本到图像生成

Efficient Personalized Text-to-image Generation by Leveraging Textual  Subspace

In this work, we systematically study the problem of personalized
text-to-image generation, where the output image is expected to portray
information about specific human subjects. E.g., generating images of oneself
appearing at imaginative places, interacting with various items, or engaging in
fictional activities. To this end, we focus on text-to-image systems that input
a single image of an individual to ground the generation process along with
text describing the desired visual context. Our first contribution is to fill
the literature gap by curating high-quality, appropriate data for this task.
Namely, we introduce a standardized dataset (Stellar) that contains
personalized prompts coupled with images of individuals that is an order of
magnitude larger than existing relevant datasets and where rich semantic
ground-truth annotations are readily available. Having established Stellar to
promote cross-systems fine-grained comparisons further, we introduce a rigorous
ensemble of specialized metrics that highlight and disentangle fundamental
properties such systems should obey. Besides being intuitive, our new metrics
correlate significantly more strongly with human judgment than currently used
metrics on this task. Last but not least, drawing inspiration from the recent
works of ELITE and SDXL, we derive a simple yet efficient, personalized
text-to-image baseline that does not require test-time fine-tuning for each
subject and which sets quantitatively and in human trials a new SoTA. For more
information, please visit our project's website:
this https URL

我们系统地研究了个性化文本生成图像的问题，提出了填补文献空白的贡献，引入了新的高质量数据集和评估指标，并提出了一个简单高效的基准模型，成为了该领域的最新技术。

Stellar: 人本个性化文本到图像方法的系统评估

Stellar: Systematic Evaluation of Human-Centric Personalized  Text-to-Image Methods

Personalized text-to-image generation has emerged as a powerful and
sought-after tool, empowering users to create customized images based on their
specific concepts and prompts. However, existing approaches to personalization
encounter multiple challenges, including long tuning times, large storage
requirements, the necessity for multiple input images per identity, and
limitations in preserving identity and editability. To address these obstacles,
we present PhotoVerse, an innovative methodology that incorporates a
dual-branch conditioning mechanism in both text and image domains, providing
effective control over the image generation process. Furthermore, we introduce
facial identity loss as a novel component to enhance the preservation of
identity during training. Remarkably, our proposed PhotoVerse eliminates the
need for test time tuning and relies solely on a single facial photo of the
target identity, significantly reducing the resource cost associated with image
generation. After a single training phase, our approach enables generating
high-quality images within only a few seconds. Moreover, our method can produce
diverse images that encompass various scenes and styles. The extensive
evaluation demonstrates the superior performance of our approach, which
achieves the dual objectives of preserving identity and facilitating
editability. Project page: this https URL

基于用户特定概念和提示创建自定义图像的个性化文本到图像生成技术已崭露头角，但现有方法在个性化方面面临多个挑战。为了解决这些障碍，我们提出了一种创新的方法 - PhotoVerse，在文本和图像领域中融入双分支条件机制，以提供对图像生成过程的有效控制，并引入了面部身份损失作为训练期间增强身份保存的新组件。这一方法消除了测试时间调整的需要，仅依赖于目标身份的单张面部照片，大大减少了与图像生成相关的资源成本。经过单一训练阶段后，我们的方法能够在几秒钟内生成高质量的图像，产生包含各种场景和风格的多样化图像。广泛的评估表明我们的方法具有卓越性能，实现了保存身份和增强可编辑性的双重目标。