Instance segmentation datasets play a crucial role in training accurate and
robust computer vision models. However, obtaining accurate mask annotations to
produce high-quality segmentation datasets is a costly and labor-intensive
process. In this work, we show how this issue can be mitigated by starting with
small annotated instance segmentation datasets and augmenting them to
effectively obtain a sizeable annotated dataset. We achieve that by creating
variations of the available annotated object instances in a way that preserves
the provided mask annotations, thereby resulting in new image-mask pairs to be
added to the set of annotated images. Specifically, we generate new images
using a diffusion-based inpainting model to fill out the masked area with a
desired object class by guiding the diffusion through the object outline. We
show that the object outline provides a simple, but also reliable and
convenient training-free guidance signal for the underlying inpainting model
that is often sufficient to fill out the mask with an object of the correct
class without further text guidance and preserve the correspondence between
generated images and the mask annotations with high precision. Our experimental
results reveal that our method successfully generates realistic variations of
object instances, preserving their shape characteristics while introducing
diversity within the augmented area. We also show that the proposed method can
naturally be combined with text guidance and other image augmentation
techniques.

使用扩充数据集的方法，通过创建基于注释目标实例的多样化图像和标记对来对实例分割数据集进行增强，同时保留原有标记的一致性。实验结果显示该方法成功生成了实际的目标实例变体，并在扩增区域内引入了多样性。同时，该方法可以与文本指导和其他图像增强技术自然地结合。

基于轮廓引导的扩散模型目标修复

Outline-Guided Object Inpainting with Diffusion Models

As online shopping is growing, the ability for buyers to virtually visualize
products in their settings-a phenomenon we define as "Virtual Try-All"-has
become crucial. Recent diffusion models inherently contain a world model,
rendering them suitable for this task within an inpainting context. However,
traditional image-conditioned diffusion models often fail to capture the
fine-grained details of products. In contrast, personalization-driven models
such as DreamPaint are good at preserving the item's details but they are not
optimized for real-time applications. We present "Diffuse to Choose," a novel
diffusion-based image-conditioned inpainting model that efficiently balances
fast inference with the retention of high-fidelity details in a given reference
item while ensuring accurate semantic manipulations in the given scene content.
Our approach is based on incorporating fine-grained features from the reference
image directly into the latent feature maps of the main diffusion model,
alongside with a perceptual loss to further preserve the reference item's
details. We conduct extensive testing on both in-house and publicly available
datasets, and show that Diffuse to Choose is superior to existing zero-shot
diffusion inpainting methods as well as few-shot diffusion personalization
algorithms like DreamPaint.

基于扩散的图像修复模型实现在线购物时的虚拟可视化，包括细节处理和语义操作。