The field of generative image inpainting and object insertion has made significant progress with the recent advent of latent diffusion models. Utilizing a precise object mask can greatly enhance these applications. However, due to the challenges users encounter in creating high-fidelity masks, there is a tendency for these methods to rely on more coarse masks (e.g., bounding box) for these applications. This results in limited control and compromised background content preservation. To overcome these limitations, we introduce SmartMask, which allows any novice user to create detailed masks for precise object insertion. Combined with a ControlNet-Inpaint model, our experiments demonstrate that SmartMask achieves superior object insertion quality, preserving the background content more effectively than previous methods. Notably, unlike prior works the proposed approach can also be used even without user-mask guidance, which allows it to perform mask-free object insertion at diverse positions and scales. Furthermore, we find that when used iteratively with a novel instruction-tuning based planning model, SmartMask can be used to design detailed layouts from scratch. As compared with user-scribble based layout design, we observe that SmartMask allows for better quality outputs with layout-to-image generation methods. Project page is available at https://smartmask-gen.github.io

通过SmartMask和ControlNet-Inpaint模型，本文介绍了一种能够实现优秀的目标插入效果并更有效地保护背景内容的方法，此方法不仅可以在用户创建详细掩码的情况下运行，还可以在无掩码条件下进行多样位置和比例的目标插入，并且还可以与基于指导的规划模型协同工作从头设计布局，相比于基于用户草图的布局设计方法，SmartMask产生更高质量的布局图像。

SmartMask：面向细粒度物体插入和布局控制的上下文感知高保真蒙版生成