Sketches make an intuitive and powerful visual expression as they are fast
executed freehand drawings. We present a method for synthesizing realistic
photos from scene sketches. Without the need for sketch and photo pairs, our
framework directly learns from readily available large-scale photo datasets in
an unsupervised manner. To this end, we introduce a standardization module that
provides pseudo sketch-photo pairs during training by converting photos and
sketches to a standardized domain, i.e. the edge map. The reduced domain gap
between sketch and photo also allows us to disentangle them into two
components: holistic scene structures and low-level visual styles such as color
and texture. Taking this advantage, we synthesize a photo-realistic image by
combining the structure of a sketch and the visual style of a reference photo.
Extensive experimental results on perceptual similarity metrics and human
perceptual studies show the proposed method could generate realistic photos
with high fidelity from scene sketches and outperform state-of-the-art photo
synthesis baselines. We also demonstrate that our framework facilitates a
controllable manipulation of photo synthesis by editing strokes of
corresponding sketches, delivering more fine-grained details than previous
approaches that rely on region-level editing.

本文提出了一种新的基于素描图的自然照片综合方法，采用无监督学习，结合标准化模块提取参考照片、在训练中提供假素描照片对，以及两种主要成分：全局场景结构和低级视觉风格的组合。实验结果表明，该方法可以生成具有高保真度的真实感照片，并且在相应场景素描的编辑方面具有更高的细节度，优于其他基于照片综合的方法。

无监督场景素描到照片的合成

Unsupervised Scene Sketch to Photo Synthesis

We present an image-based VIirtual Try-On Network (VITON) without using 3D
information in any form, which seamlessly transfers a desired clothing item
onto the corresponding region of a person using a coarse-to-fine strategy.
Conditioned upon a new clothing-agnostic yet descriptive person representation,
our framework first generates a coarse synthesized image with the target
clothing item overlaid on that same person in the same pose. We further enhance
the initial blurry clothing area with a refinement network. The network is
trained to learn how much detail to utilize from the target clothing item, and
where to apply to the person in order to synthesize a photo-realistic image in
which the target item deforms naturally with clear visual patterns. Experiments
on our newly collected Zalando dataset demonstrate its promise in the
image-based virtual try-on task over state-of-the-art generative models.

本研究提出了一个基于图像的 Virtual Try-On Network，采用 “粗到细” 的策略将所需的服装项目无缝地转移到相应的人体部位，从而合成真实的照片，不需要 3D 信息，并通过细化网络训练来改善初始模糊图像。