Previous researches of sketches often considered sketches in pixel format and
leveraged CNN based models in the sketch understanding. Fundamentally, a sketch
is stored as a sequence of data points, a vector format representation, rather
than the photo-realistic image of pixels. SketchRNN studied a generative neural
representation for sketches of vector format by Long Short Term Memory networks
(LSTM). Unfortunately, the representation learned by SketchRNN is primarily for
the generation tasks, rather than the other tasks of recognition and retrieval
of sketches. To this end and inspired by the recent BERT model, we present a
model of learning Sketch Bidirectional Encoder Representation from Transformer
(Sketch-BERT). We generalize BERT to sketch domain, with the novel proposed
components and pre-training algorithms, including the newly designed sketch
embedding networks, and the self-supervised learning of sketch gestalt.
Particularly, towards the pre-training task, we present a novel Sketch Gestalt
Model (SGM) to help train the Sketch-BERT. Experimentally, we show that the
learned representation of Sketch-BERT can help and improve the performance of
the downstream tasks of sketch recognition, sketch retrieval, and sketch
gestalt.

该研究提出了一种基于 Transformer 的 sketch 领域学习模型 Sketch-BERT，包括新设计的 sketch 嵌入网络和自监督学习的 sketch gestalt，其在 pre-training 和 downstream tasks 中提高了 sketch 识别、检索、以及 gestalt 任务的表现。