Spatially-adaptive normalization (SPADE) is remarkably successful recently in
conditional semantic image synthesis \cite{park2019semantic}, which modulates
the normalized activation with spatially-varying transformations learned from
semantic layouts, to prevent the semantic information from being washed away.
Despite its impressive performance, a more thorough understanding of the
advantages inside the box is still highly demanded to help reduce the
significant computation and parameter overhead introduced by this novel
structure. In this paper, from a return-on-investment point of view, we conduct
an in-depth analysis of the effectiveness of this spatially-adaptive
normalization and observe that its modulation parameters benefit more from
semantic-awareness rather than spatial-adaptiveness, especially for
high-resolution input masks. Inspired by this observation, we propose
class-adaptive normalization (CLADE), a lightweight but equally-effective
variant that is only adaptive to semantic class. In order to further improve
spatial-adaptiveness, we introduce intra-class positional map encoding
calculated from semantic layouts to modulate the normalization parameters of
CLADE and propose a truly spatially-adaptive variant of CLADE, namely
CLADE-ICPE.Through extensive experiments on multiple challenging datasets, we
demonstrate that the proposed CLADE can be generalized to different SPADE-based
methods while achieving comparable generation quality compared to SPADE, but it
is much more efficient with fewer extra parameters and lower computational
cost. The code and pretrained models are available at
https://github.com/tzt101/CLADE.git.

本文从高回报的角度，对空间自适应规范化进行了深入分析，发现其调制参数更加受到语义感知而非空间自适应的影响。在此观察的启发下，本文提出了一种轻量级但同样有效的类自适应规范化变体，称为 CLADE，并介绍了一种计算自语义布局的类内位置图编码，以调制 CLADE 的规范化参数，提出了一种真正的空间自适应变体，即 CLADE-ICPE。通过在多个具有挑战性的数据集上进行大量实验证明，所提出的 CLADE 方法可以推广到不同的基于 SPADE 的方法，并实现与 SPADE 相当的生成质量，而且具有更少的额外参数和更低的计算成本。

通过类适应归一化实现高效语义图像合成

Efficient Semantic Image Synthesis via Class-Adaptive Normalization

Spatially-adaptive normalization is remarkably successful recently in
conditional semantic image synthesis, which modulates the normalized activation
with spatially-varying transformations learned from semantic layouts, to
preserve the semantic information from being washed away. Despite its
impressive performance, a more thorough understanding of the true advantages
inside the box is still highly demanded, to help reduce the significant
computation and parameter overheads introduced by these new structures. In this
paper, from a return-on-investment point of view, we present a deep analysis of
the effectiveness of SPADE and observe that its advantages actually come mainly
from its semantic-awareness rather than the spatial-adaptiveness. Inspired by
this point, we propose class-adaptive normalization (CLADE), a lightweight
variant that is not adaptive to spatial positions or layouts. Benefited from
this design, CLADE greatly reduces the computation cost while still being able
to preserve the semantic information during the generation. Extensive
experiments on multiple challenging datasets demonstrate that while the
resulting fidelity is on par with SPADE, its overhead is much cheaper than
SPADE. Take the generator for ADE20k dataset as an example, the extra parameter
and computation cost introduced by CLADE are only 4.57% and 0.07% while that of
SPADE are 39.21% and 234.73% respectively.

通过 ROI 分析，我们发现在 SPA 上的优势主要来自于其对语义信息的敏感性，而不是其对适应性位置的适应性。 因此，我们提出了一个轻量级变种 CLADE，它不适应空间位置或布局。 CLADE 极大地降低了计算成本，同时仍然能够在生成过程中保持语义信息。

重新思考空间自适应归一化

Rethinking Spatially-Adaptive Normalization

We propose spatially-adaptive normalization, a simple but effective layer for
synthesizing photorealistic images given an input semantic layout. Previous
methods directly feed the semantic layout as input to the deep network, which
is then processed through stacks of convolution, normalization, and
nonlinearity layers. We show that this is suboptimal as the normalization
layers tend to ``wash away'' semantic information. To address the issue, we
propose using the input layout for modulating the activations in normalization
layers through a spatially-adaptive, learned transformation. Experiments on
several challenging datasets demonstrate the advantage of the proposed method
over existing approaches, regarding both visual fidelity and alignment with
input layouts. Finally, our model allows user control over both semantic and
style. Code is available at this https URL .

通过使用自适应规范化层来调整激活函数，我们提出了一种简单但有效的方法来合成具有输入语义布局的逼真图像，这种方法可以提高视觉保真度和与输入布局的对齐度，并允许用户控制语义和风格。