Diffusion-based technologies have made significant strides, particularly in
personalized and customized facialgeneration. However, existing methods face
challenges in achieving high-fidelity and detailed identity (ID)consistency,
primarily due to insufficient fine-grained control over facial areas and the
lack of a comprehensive strategy for ID preservation by fully considering
intricate facial details and the overall face. To address these limitations, we
introduce ConsistentID, an innovative method crafted for
diverseidentity-preserving portrait generation under fine-grained multimodal
facial prompts, utilizing only a single reference image. ConsistentID comprises
two key components: a multimodal facial prompt generator that combines facial
features, corresponding facial descriptions and the overall facial context to
enhance precision in facial details, and an ID-preservation network optimized
through the facial attention localization strategy, aimed at preserving ID
consistency in facial regions. Together, these components significantly enhance
the accuracy of ID preservation by introducing fine-grained multimodal ID
information from facial regions. To facilitate training of ConsistentID, we
present a fine-grained portrait dataset, FGID, with over 500,000 facial images,
offering greater diversity and comprehensiveness than existing public facial
datasets. % such as LAION-Face, CelebA, FFHQ, and SFHQ. Experimental results
substantiate that our ConsistentID achieves exceptional precision and diversity
in personalized facial generation, surpassing existing methods in the MyStyle
dataset. Furthermore, while ConsistentID introduces more multimodal ID
information, it maintains a fast inference speed during generation.

ConsistentID 是一种创新方法，用于在细粒度多模式面部提示下生成多样性身份保留肖像，利用仅一个参考图像，通过组合面部特征、相应的面部描述和整体面部上下文来提高面部细节的精度，并通过面部注意力定位策略来优化身份保留网络，旨在保持面部区域的身份一致性。

ConsistentID: 多模态细粒度身份保持的人像生成

ConsistentID: Portrait Generation with Multimodal Fine-Grained Identity  Preserving

Despite the significant progress in recent years, very few of the AI-based
talking face generation methods attempt to render natural emotions. Moreover,
the scope of the methods is majorly limited to the characteristics of the
training dataset, hence they fail to generalize to arbitrary unseen faces. In
this paper, we propose a one-shot facial geometry-aware emotional talking face
generation method that can generalize to arbitrary faces. We propose a graph
convolutional neural network that uses speech content feature, along with an
independent emotion input to generate emotion and speech-induced motion on
facial geometry-aware landmark representation. This representation is further
used in our optical flow-guided texture generation network for producing the
texture. We propose a two-branch texture generation network, with motion and
texture branches designed to consider the motion and texture content
independently. Compared to the previous emotion talking face methods, our
method can adapt to arbitrary faces captured in-the-wild by fine-tuning with
only a single image of the target identity in neutral emotion.

本文提出了一种面部生成方法，使用基于语音内容特征的图卷积神经网络，结合独立的情感输入，生成面部几何感知标记表示上的情感和语音感应运动，并在此基础上，利用光流引导的纹理生成网络生成纹理。