In this paper, we delve into a new task known as small object editing (SOE),
which focuses on text-based image inpainting within a constrained, small-sized
area. Despite the remarkable success have been achieved by current image
inpainting approaches, their application to the SOE task generally results in
failure cases such as Object Missing, Text-Image Mismatch, and Distortion.
These failures stem from the limited use of small-sized objects in training
datasets and the downsampling operations employed by U-Net models, which
hinders accurate generation. To overcome these challenges, we introduce a novel
training-based approach, SOEDiff, aimed at enhancing the capability of baseline
models like StableDiffusion in editing small-sized objects while minimizing
training costs. Specifically, our method involves two key components: SO-LoRA,
which efficiently fine-tunes low-rank matrices, and Cross-Scale Score
Distillation loss, which leverages high-resolution predictions from the
pre-trained teacher diffusion model. Our method presents significant
improvements on the test dataset collected from MSCOCO and OpenImage,
validating the effectiveness of our proposed method in small object editing. In
particular, when comparing SOEDiff with SD-I model on the OpenImage-f dataset,
we observe a 0.99 improvement in CLIP-Score and a reduction of 2.87 in FID. Our
project page can be found in this https URL

本文探讨了一个名为小物体编辑（SOE）的新任务，重点是在受限制的小尺寸区域内进行基于文本的图像修复。我们引入了一种新颖的以训练为基础的方法 SOEDiff，用于增强基线模型的能力，同时最小化训练成本，这包括高效地微调低秩矩阵和利用预训练教师扩散模型的高分辨率预测来提升交叉尺度得分蒸馏损失。通过在 MSCOCO 和 OpenImage 收集的测试数据集上进行实验证明了我们所提出的小物体编辑方法的有效性。与 OpenImage-f 数据集上的 SD-I 模型相比，我们观察到在 CLIP-Score 上的提升为 0.99，FID 的减少为 2.87。