There has been significant progress in personalized image synthesis with
methods such as Textual Inversion, DreamBooth, and LoRA. Yet, their real-world
applicability is hindered by high storage demands, lengthy fine-tuning
processes, and the need for multiple reference images. Conversely, existing ID
embedding-based methods, while requiring only a single forward inference, face
challenges: they either necessitate extensive fine-tuning across numerous model
parameters, lack compatibility with community pre-trained models, or fail to
maintain high face fidelity. Addressing these limitations, we introduce
InstantID, a powerful diffusion model-based solution. Our plug-and-play module
adeptly handles image personalization in various styles using just a single
facial image, while ensuring high fidelity. To achieve this, we design a novel
IdentityNet by imposing strong semantic and weak spatial conditions,
integrating facial and landmark images with textual prompts to steer the image
generation. InstantID demonstrates exceptional performance and efficiency,
proving highly beneficial in real-world applications where identity
preservation is paramount. Moreover, our work seamlessly integrates with
popular pre-trained text-to-image diffusion models like SD1.5 and SDXL, serving
as an adaptable plugin. Our codes and pre-trained checkpoints will be available
at this https URL

我们介绍了基于 InstantID 的个性化图像合成方法，通过应用 IdentityNet 模型，使用单张面部图像实现各种风格的图像个性化，同时确保高保真度，并与 SD1.5 和 SDXL 等常用预训练文本到图像扩散模型无缝集成，具有卓越的性能和高效性，在重视身份保护的实际应用中极具益处。

InstantID: 秒级零样本保身份生成

InstantID: Zero-shot Identity-Preserving Generation in Seconds

Recent advances in personalized image generation allow a pre-trained
text-to-image model to learn a new concept from a set of images. However,
existing personalization approaches usually require heavy test-time finetuning
for each concept, which is time-consuming and difficult to scale. We propose
InstantBooth, a novel approach built upon pre-trained text-to-image models that
enables instant text-guided image personalization without any test-time
finetuning. We achieve this with several major components. First, we learn the
general concept of the input images by converting them to a textual token with
a learnable image encoder. Second, to keep the fine details of the identity, we
learn rich visual feature representation by introducing a few adapter layers to
the pre-trained model. We train our components only on text-image pairs without
using paired images of the same concept. Compared to test-time finetuning-based
methods like DreamBooth and Textual-Inversion, our model can generate
competitive results on unseen concepts concerning language-image alignment,
image fidelity, and identity preservation while being 100 times faster.

提出了一种名为 InstantBooth 的方法，该方法通过学习图像概念的文本标记和适配层，无需进行测试时间微调即可实现文本引导图像个性化生成，并在语言 - 图像对齐、图像保真度和身份保护等方面获得了竞争性结果，速度快 100 倍。