Generative Adversarial Networks (GANs) have recently achieved significant
improvement on paired/unpaired image-to-image translation, such as
photo$\rightarrow$ sketch and artist painting style transfer. However, existing
models can only be capable of transferring the low-level information (e.g.
color or texture changes), but fail to edit high-level semantic meanings (e.g.,
geometric structure or content) of objects. On the other hand, while some
researches can synthesize compelling real-world images given a class label or
caption, they cannot condition on arbitrary shapes or structures, which largely
limits their application scenarios and interpretive capability of model
results. In this work, we focus on a more challenging semantic manipulation
task, which aims to modify the semantic meaning of an object while preserving
its own characteristics (e.g. viewpoints and shapes), such as
cow$\rightarrow$sheep, motor$\rightarrow$ bicycle, cat$\rightarrow$dog. To
tackle such large semantic changes, we introduce a contrasting GAN
(contrast-GAN) with a novel adversarial contrasting objective. Instead of
directly making the synthesized samples close to target data as previous GANs
did, our adversarial contrasting objective optimizes over the distance
comparisons between samples, that is, enforcing the manipulated data be
semantically closer to the real data with target category than the input data.
Equipped with the new contrasting objective, a novel mask-conditional
contrast-GAN architecture is proposed to enable disentangle image background
with object semantic changes. Experiments on several semantic manipulation
tasks on ImageNet and MSCOCO dataset show considerable performance gain by our
contrast-GAN over other conditional GANs. Quantitative results further
demonstrate the superiority of our model on generating manipulated results with
high visual fidelity and reasonable object semantics.

本文提出了一种对物体进行语义操纵的新的生成对抗网络方法，使用对比对抗目标而不是直接将合成样本接近目标数据，以生成具有高可视度和合理的对象语义的操作结果，并在多个语义操作任务上展示了我们的模型的优越性。