Humans show high-level of abstraction capabilities in games that require
quickly communicating object information. They decompose the message content
into multiple parts and communicate them in an interpretable protocol. Toward
equipping machines with such capabilities, we propose the Primitive-based
Sketch Abstraction task where the goal is to represent sketches using a fixed
set of drawing primitives under the influence of a budget. To solve this task,
our Primitive-Matching Network (PMN), learns interpretable abstractions of a
sketch in a self supervised manner. Specifically, PMN maps each stroke of a
sketch to its most similar primitive in a given set, predicting an affine
transformation that aligns the selected primitive to the target stroke. We
learn this stroke-to-primitive mapping end-to-end with a distance-transform
loss that is minimal when the original sketch is precisely reconstructed with
the predicted primitives. Our PMN abstraction empirically achieves the highest
performance on sketch recognition and sketch-based image retrieval given a
communication budget, while at the same time being highly interpretable. This
opens up new possibilities for sketch analysis, such as comparing sketches by
extracting the most relevant primitives that define an object category. Code is
available at this https URL.

该研究提出了基于原语的素描抽象任务，通过 Primitive-Matching Network（PMN）学习解释性抽象，将每个笔画映射到其最相似的原语，并实现了用于笔画识别的最高性能和高度可解释性的 PMN 抽象。