We present CoGS, a novel method for the style-conditioned, sketch-driven
synthesis of images. CoGS enables exploration of diverse appearance
possibilities for a given sketched object, enabling decoupled control over the
structure and the appearance of the output. Coarse-grained control over object
structure and appearance are enabled via an input sketch and an exemplar
"style" conditioning image to a transformer-based sketch and style encoder to
generate a discrete codebook representation. We map the codebook representation
into a metric space, enabling fine-grained control over selection and
interpolation between multiple synthesis options before generating the image
via a vector quantized GAN (VQGAN) decoder. Our framework thereby unifies
search and synthesis tasks, in that a sketch and style pair may be used to run
an initial synthesis which may be refined via combination with similar results
in a search corpus to produce an image more closely matching the user's intent.
We show that our model, trained on the 125 object classes of our newly created
Pseudosketches dataset, is capable of producing a diverse gamut of semantic
content and appearance styles.

使用 CoGS 方法，通过输入手绘草图和样式编码图像生成图像，实现对图像结构和外观的粗粒度控制；进一步通过将码簿表示映射到向量空间，实现对多个综合选项的精细控制；同时，该方法还融合了搜索和综合任务，使用户可以根据搜索结果进一步优化图像的细节。