Quantization has emerged as one of the most prevalent approaches to compress
and accelerate neural networks. Recently, data-free quantization has been
widely studied as a practical and promising solution. It synthesizes data for
calibrating the quantized model according to the batch normalization (BN)
statistics of FP32 ones and significantly relieves the heavy dependency on real
training data in traditional quantization methods. Unfortunately, we find that
in practice, the synthetic data identically constrained by BN statistics
suffers serious homogenization at both distribution level and sample level and
further causes a significant performance drop of the quantized model. We
propose Diverse Sample Generation (DSG) scheme to mitigate the adverse effects
caused by homogenization. Specifically, we slack the alignment of feature
statistics in the BN layer to relax the constraint at the distribution level
and design a layerwise enhancement to reinforce specific layers for different
data samples. Our DSG scheme is versatile and even able to be applied to the
state-of-the-art post-training quantization method like AdaRound. We evaluate
the DSG scheme on the large-scale image classification task and consistently
obtain significant improvements over various network architectures and
quantization methods, especially when quantized to lower bits (e.g., up to 22%
improvement on W4A4). Moreover, benefiting from the enhanced diversity, models
calibrated by synthetic data perform close to those calibrated by real data and
even outperform them on W4A4.

数据无关量化方法的统计数据不均衡问题限制了其在神经网络压缩方面的应用；本文提出了 Diverse Sample Generation（DSG）方法以在分布和样本层面缓解这一问题，其可以应用于当前一些最先进的量化方法中，如 AdaRound。在大规模图像分类任务上，经过合成数据校准的模型的性能可以达到通过真实数据校准的模型的性能，甚至在一些情况下表现更好。

多样化样本生成以获得精准的无数据量化

Diversifying Sample Generation for Accurate Data-Free Quantization

Self-supervised (SS) learning is a powerful approach for representation
learning using unlabeled data. Recently, it has been applied to Generative
Adversarial Networks (GAN) training. Specifically, SS tasks were proposed to
address the catastrophic forgetting issue in the GAN discriminator. In this
work, we perform an in-depth analysis to understand how SS tasks interact with
learning of generator. From the analysis, we identify issues of SS tasks which
allow a severely mode-collapsed generator to excel the SS tasks. To address the
issues, we propose new SS tasks based on a multi-class minimax game. The
competition between our proposed SS tasks in the game encourages the generator
to learn the data distribution and generate diverse samples. We provide both
theoretical and empirical analysis to support that our proposed SS tasks have
better convergence property. We conduct experiments to incorporate our proposed
SS tasks into two different GAN baseline models. Our approach establishes
state-of-the-art FID scores on CIFAR-10, CIFAR-100, STL-10, CelebA, Imagenet
$32\times32$ and Stacked-MNIST datasets, outperforming existing works by
considerable margins in some cases. Our unconditional GAN model approaches
performance of conditional GAN without using labeled data. Our code:
this https URL

本文针对 GAN 模型中的模式崩溃问题，提出了基于多类极小极大博弈的自监督任务。实验表明，该方法在多个数据集上表现出了优异的性能，创造了良好的状态。

自监督 GAN：多类极小极大博弈分析和改进

Self-supervised GAN: Analysis and Improvement with Multi-class Minimax  Game

We propose MAD-GAN, an intuitive generalization to the Generative Adversarial
Networks (GANs) and its conditional variants to address the well known problem
of mode collapse. First, MAD-GAN is a multi-agent GAN architecture
incorporating multiple generators and one discriminator. Second, to enforce
that different generators capture diverse high probability modes, the
discriminator of MAD-GAN is designed such that along with finding the real and
fake samples, it is also required to identify the generator that generated the
given fake sample. Intuitively, to succeed in this task, the discriminator must
learn to push different generators towards different identifiable modes. We
perform extensive experiments on synthetic and real datasets and compare
MAD-GAN with different variants of GAN. We show high quality diverse sample
generations for challenging tasks such as image-to-image translation and face
generation. In addition, we also show that MAD-GAN is able to disentangle
different modalities when trained using highly challenging diverse-class
dataset (e.g. dataset with images of forests, icebergs, and bedrooms). In the
end, we show its efficacy on the unsupervised feature representation task. In
Appendix, we introduce a similarity based competing objective (MAD-GAN-Sim)
which encourages different generators to generate diverse samples based on a
user defined similarity metric. We show its performance on the image-to-image
translation, and also show its effectiveness on the unsupervised feature
representation task.

介绍一种名为 MAD-GAN 的多智能体生成对抗网络架构，可以解决 GAN 领域的模式崩溃问题，实验表明该架构能产生高质量的样本且具有较强的去卷积能力。