We introduce a novel sketch-to-image tool that aligns with the iterative refinement process of artists. Our tool lets users sketch blocking strokes to coarsely represent the placement and form of objects and detail strokes to refine their shape and silhouettes. We develop a two-pass algorithm for generating high-fidelity images from such sketches at any point in the iterative process. In the first pass we use a ControlNet to generate an image that strictly follows all the strokes (blocking and detail) and in the second pass we add variation by renoising regions surrounding blocking strokes. We also present a dataset generation scheme that, when used to train a ControlNet architecture, allows regions that do not contain strokes to be interpreted as not-yet-specified regions rather than empty space. We show that this partial-sketch-aware ControlNet can generate coherent elements from partial sketches that only contain a small number of strokes. The high-fidelity images produced by our approach serve as scaffolds that can help the user adjust the shape and proportions of objects or add additional elements to the composition. We demonstrate the effectiveness of our approach with a variety of examples and evaluative comparisons.

我们介绍了一种与艺术家的迭代细化过程相适应的新型素描到图像工具。我们的工具允许用户绘制出粗略表示物体放置和形状的阻塞笔触以及用于精炼形状和轮廓的详细笔触。我们开发了一个两步算法，从这些素描中的任意时刻生成高保真度图像。第一步中，我们使用ControlNet生成完全遵循所有笔触（阻塞笔触和详细笔触）的图像，第二步通过给阻塞笔触周围的区域添加噪声来增加变异性。我们还提出了一种数据集生成方案，用于训练ControlNet架构，使不包含笔触的区域被解释为尚未指定的区域而不是空白空间。我们展示了这种部分素描感知的ControlNet能够从仅含少量笔触的部分素描中生成连贯的元素。我们的方法生成的高保真度图像可以作为脚手架，帮助用户调整物体的形状和比例，或者在构图中添加额外的元素。我们通过各种例子和评估性比较证明了我们方法的有效性。

区块与细节：搭建草图到图像生成系统