We present a method for expanding a dataset by incorporating knowledge from the wide distribution of pre-trained latent diffusion models. Data augmentations typically incorporate inductive biases about the image formation process into the training (e.g. translation, scaling, colour changes, etc.). Here, we go beyond simple pixel transformations and introduce the concept of instance-level data augmentation by repainting parts of the image at the level of object instances. The method combines a conditional diffusion model with depth and edge maps control conditioning to seamlessly repaint individual objects inside the scene, being applicable to any segmentation or detection dataset. Used as a data augmentation method, it improves the performance and generalization of the state-of-the-art salient object detection, semantic segmentation and object detection models. By redrawing all privacy-sensitive instances (people, license plates, etc.), the method is also applicable for data anonymization. We also release fully synthetic and anonymized expansions for popular datasets: COCO, Pascal VOC and DUTS.

本研究提出了一种使用预训练的潜在扩散模型来扩充数据集的方法，并引入了实例级数据增强的概念。该方法通过重新绘制场景内的个别对象来改善卓越的显著目标检测、语义分割和物体检测模型的性能和泛化能力，并在需要数据匿名化时，通过重新绘制隐私敏感实例来实现。

数据集增强与实例级别的数据扩充