Deep image-based modeling received lots of attention in recent years, yet the
parallel problem of sketch-based modeling has only been briefly studied, often
as a potential application. In this work, for the first time, we identify the
main differences between sketch and image inputs: (i) style variance, (ii)
imprecise perspective, and (iii) sparsity. We discuss why each of these
differences can pose a challenge, and even make a certain class of image-based
methods inapplicable. We study alternative solutions to address each of the
difference. By doing so, we drive out a few important insights: (i) sparsity
commonly results in an incorrect prediction of foreground versus background,
(ii) diversity of human styles, if not taken into account, can lead to very
poor generalization properties, and finally (iii) unless a dedicated sketching
interface is used, one can not expect sketches to match a perspective of a
fixed viewpoint. Finally, we compare a set of representative deep single-image
modeling solutions and show how their performance can be improved to tackle
sketch input by taking into consideration the identified critical differences.

该研究主要研究了手绘草图和图片输入之间的区别和挑战，并提出了改进单一图片建模算法以适用于手绘草图的解决方案。

深度基于草图建模：技巧和窍门

Deep Sketch-Based Modeling: Tips and Tricks

Sketch-based modeling strives to bring the ease and immediacy of drawing to
the 3D world. However, while drawings are easy for humans to create, they are
very challenging for computers to interpret due to their sparsity and
ambiguity. We propose a data-driven approach that tackles this challenge by
learning to reconstruct 3D shapes from one or more drawings. At the core of our
approach is a deep convolutional neural network (CNN) that predicts occupancy
of a voxel grid from a line drawing. This CNN provides us with an initial 3D
reconstruction as soon as the user completes a single drawing of the desired
shape. We complement this single-view network with an updater CNN that refines
an existing prediction given a new drawing of the shape created from a novel
viewpoint. A key advantage of our approach is that we can apply the updater
iteratively to fuse information from an arbitrary number of viewpoints, without
requiring explicit stroke correspondences between the drawings. We train both
CNNs by rendering synthetic contour drawings from hand-modeled shape
collections as well as from procedurally-generated abstract shapes. Finally, we
integrate our CNNs in a minimal modeling interface that allows users to
seamlessly draw an object, rotate it to see its 3D reconstruction, and refine
it by re-drawing from another vantage point using the 3D reconstruction as
guidance. The main strengths of our approach are its robustness to freehand
bitmap drawings, its ability to adapt to different object categories, and the
continuum it offers between single-view and multi-view sketch-based modeling.

我们提出了一种数据驱动方法，利用卷积神经网络 (CNN) 从一个或多个涂鸦中学习重建 3D 形状，为用户提供最初的 3D 重建并可以通过应用更新器 CNN 来迭代更新和融合多视角信息，从而实现自由的位图创作和多视角草图建模之间的连续转换。