We introduce the Multi-Instance Generation (MIG) task, which focuses on
generating multiple instances within a single image, each accurately placed at
predefined positions with attributes such as category, color, and shape,
strictly following user specifications. MIG faces three main challenges:
avoiding attribute leakage between instances, supporting diverse instance
descriptions, and maintaining consistency in iterative generation. To address
attribute leakage, we propose the Multi-Instance Generation Controller (MIGC).
MIGC generates multiple instances through a divide-and-conquer strategy,
breaking down multi-instance shading into single-instance tasks with singular
attributes, later integrated. To provide more types of instance descriptions,
we developed MIGC++. MIGC++ allows attribute control through text \& images and
position control through boxes \& masks. Lastly, we introduced the
Consistent-MIG algorithm to enhance the iterative MIG ability of MIGC and
MIGC++. This algorithm ensures consistency in unmodified regions during the
addition, deletion, or modification of instances, and preserves the identity of
instances when their attributes are changed. We introduce the COCO-MIG and
Multimodal-MIG benchmarks to evaluate these methods. Extensive experiments on
these benchmarks, along with the COCO-Position benchmark and DrawBench,
demonstrate that our methods substantially outperform existing techniques,
maintaining precise control over aspects including position, attribute, and
quantity. Project page: this https URL

我们介绍了多实例生成（Multi-Instance Generation，MIG）任务，着重于在单个图像内生成多个实例，每个实例都根据用户的要求准确地放置在预定义位置，并具有类别、颜色和形状等属性。MIG 面临三个主要挑战：避免实例之间的属性泄漏，支持多样化的实例描述，并在迭代生成过程中保持一致性。为了解决属性泄漏问题，我们提出了多实例生成控制器（Multi-Instance Generation Controller，MIGC）。MIGC 通过分而治之的策略生成多个实例，将多实例阴影拆分为具有单一属性的单实例任务，然后进行整合。为了提供更多类型的实例描述，我们开发了 MIGC++。MIGC++ 允许通过文本和图像进行属性控制，并通过框和掩码进行位置控制。最后，我们介绍了一种称为 “一致性 MIG” 算法来增强 MIGC 和 MIGC++ 的迭代 MIG 能力。该算法在添加、删除或修改实例的过程中保证未修改区域的一致性，并在实例的属性改变时保持其身份。我们引入了 COCO-MIG 和 Multimodal-MIG 基准来评估这些方法。对这些基准进行了大量实验证明我们的方法在位置、属性和数量等方面明显优于现有技术，能够精确控制多个方面。

MIGC++：图像合成高级多实例生成控制器

MIGC++: Advanced Multi-Instance Generation Controller for Image  Synthesis

Face swapping aims at injecting a source image's identity (i.e., facial
features) into a target image, while strictly preserving the target's
attributes, which are irrelevant to identity. However, we observed that
previous approaches still suffer from source attribute leakage, where the
source image's attributes interfere with the target image's. In this paper, we
analyze the latent space of StyleGAN and find the adequate combination of the
latents geared for face swapping task. Based on the findings, we develop a
simple yet robust face swapping model, RobustSwap, which is resistant to the
potential source attribute leakage. Moreover, we exploit the coordination of
3DMM's implicit and explicit information as a guidance to incorporate the
structure of the source image and the precise pose of the target image. Despite
our method solely utilizing an image dataset without identity labels for
training, our model has the capability to generate high-fidelity and temporally
consistent videos. Through extensive qualitative and quantitative evaluations,
we demonstrate that our method shows significant improvements compared with the
previous face swapping models in synthesizing both images and videos. Project
page is available at this https URL

本研究为了解决面部交换过程中源属性泄露的问题，通过分析 StyleGAN 的潜在空间，设计出一种名为 RobustSwap 的面部交换模型，同时利用 3DMM 的内隐和显性信息辅助实现源图像结构和目标图像姿态的精确匹配，最终通过定量和定性的实验表明 RobustSwap 模型在生成高保真和时空连贯图像和视频的能力方面优于先前的面部交换模型。