Recently introduced ControlNet has the ability to steer the text-driven image
generation process with geometric input such as human 2D pose, or edge
features. While ControlNet provides control over the geometric form of the
instances in the generated image, it lacks the capability to dictate the visual
appearance of each instance. We present FineControlNet to provide fine control
over each instance's appearance while maintaining the precise pose control
capability. Specifically, we develop and demonstrate FineControlNet with
geometric control via human pose images and appearance control via
instance-level text prompts. The spatial alignment of instance-specific text
prompts and 2D poses in latent space enables the fine control capabilities of
FineControlNet. We evaluate the performance of FineControlNet with rigorous
comparison against state-of-the-art pose-conditioned text-to-image diffusion
models. FineControlNet achieves superior performance in generating images that
follow the user-provided instance-specific text prompts and poses compared with
existing methods. Project webpage:
this https URL

通过人体姿势图片的几何控制和实例级别的文本提示，FineControlNet 实现了每个实例外观的精细控制，同时保持精确的姿势控制能力，相对于现有方法，FineControlNet 在生成遵循用户提供的实例特定文本提示和姿势的图像方面表现出卓越性能。