Personalization techniques for large text-to-image (T2I) models allow users
to incorporate new concepts from reference images. However, existing methods
primarily rely on textual descriptions, leading to limited control over
customized images and failing to support fine-grained and local editing (e.g.,
shape, pose, and details). In this paper, we identify sketches as an intuitive
and versatile representation that can facilitate such control, e.g., contour
lines capturing shape information and flow lines representing texture. This
motivates us to explore a novel task of sketch concept extraction: given one or
more sketch-image pairs, we aim to extract a special sketch concept that
bridges the correspondence between the images and sketches, thus enabling
sketch-based image synthesis and editing at a fine-grained level. To accomplish
this, we introduce CustomSketching, a two-stage framework for extracting novel
sketch concepts. Considering that an object can often be depicted by a contour
for general shapes and additional strokes for internal details, we introduce a
dual-sketch representation to reduce the inherent ambiguity in sketch
depiction. We employ a shape loss and a regularization loss to balance fidelity
and editability during optimization. Through extensive experiments, a user
study, and several applications, we show our method is effective and superior
to the adapted baselines.

使用个性化技术的大型文本到图像（T2I）模型允许用户从参考图像中融入新概念。本文旨在通过探索一种新颖的任务，即草图概念提取，通过两阶段框架 CustomSketching 实现草图概念的提取，以实现基于草图的图像合成和编辑。