Generative adversarial networks (GANs) are known to benefit from regularization or normalization of their critic (discriminator) network during training. In this paper, we analyze the popular spectral normalization scheme, find a significant drawback and introduce sparsity aware normalization (SAN), a new alternative approach for stabilizing GAN training. As opposed to other normalization methods, our approach explicitly accounts for the sparse nature of the feature maps in convolutional networks with ReLU activations. We illustrate the effectiveness of our method through extensive experiments with a variety of network architectures. As we show, sparsity is particularly dominant in critics used for image-to-image translation settings. In these cases our approach improves upon existing methods, in less training epochs and with smaller capacity networks, while requiring practically no computational overhead.

本文提出了一种新的归一化方法（SAN），并通过大量实验证明，与现有的方法相比，SAN能够在GAN的训练中取得更好的效果。SAN考虑了稀疏性，并且在ReLU激活的卷积网络中特别有效，并且在图像到图像翻译任务中表现出更好的性能，并且能够在较少的训练时期内以及较小的容量网络中发挥作用，而且几乎不需要计算开销。

GAN的稀疏感知归一化