A discriminatively trained neural net classifier can fit the training data
perfectly if all information about its input other than class membership has
been discarded prior to the output layer. Surprisingly, past research has
discovered that some extraneous visual detail remains in the logit vector. This
finding is based on inversion techniques that map deep embeddings back to
images. We explore this phenomenon further using a novel synthesis of methods,
yielding a feedforward inversion model that produces remarkably high fidelity
reconstructions, qualitatively superior to those of past efforts. When applied
to an adversarially robust classifier model, the reconstructions contain
sufficient local detail and global structure that they might be confused with
the original image in a quick glance, and the object category can clearly be
gleaned from the reconstruction. Our approach is based on BigGAN (Brock, 2019),
with conditioning on logits instead of one-hot class labels. We use our
reconstruction model as a tool for exploring the nature of representations,
including: the influence of model architecture and training objectives
(specifically robust losses), the forms of invariance that networks achieve,
representational differences between correctly and incorrectly classified
images, and the effects of manipulating logits and images. We believe that our
method can inspire future investigations into the nature of information flow in
a neural net and can provide diagnostics for improving discriminative models.

本篇文章研究神经网络分类器的信息流动及其影响因素，包括模型结构，训练目标（特别是鲁棒性损失），正确和错误分类图像之间的表征差异，以及对 logits 和图像进行操作的效果，提出了一种基于 BigGAN 的前馈反演模型，可以生成高保真重建的图像。

通过前馈反演经过区分训练的分类器理解不变性

Understanding Invariance via Feedforward Inversion of Discriminatively  Trained Classifiers

Generative Adversarial Networks (GANs) have become a powerful approach for
generative image modeling. However, GANs are notorious for their training
instability, especially on large-scale, complex datasets. While the recent work
of BigGAN has significantly improved the quality of image generation on
ImageNet, it requires a huge model, making it hard to deploy on
resource-constrained devices. To reduce the model size, we propose a black-box
knowledge distillation framework for compressing GANs, which highlights a
stable and efficient training process. Given BigGAN as the teacher network, we
manage to train a much smaller student network to mimic its functionality,
achieving competitive performance on Inception and FID scores with the
generator having $16\times$ fewer parameters.

提出了一种基于黑盒知识蒸馏的 GAN 模型压缩方法，将 BigGAN 作为教师网络，用较少的参数训练学生网络以模拟其功能，有效地缩小模型规模并在图像生成方面取得了有竞争力的表现。

TinyGAN：基于 BigGAN 的有条件图像生成蒸馏

TinyGAN: Distilling BigGAN for Conditional Image Generation

The ability to automatically estimate the quality and coverage of the samples
produced by a generative model is a vital requirement for driving algorithm
research. We present an evaluation metric that can separately and reliably
measure both of these aspects in image generation tasks by forming explicit,
non-parametric representations of the manifolds of real and generated data. We
demonstrate the effectiveness of our metric in StyleGAN and BigGAN by providing
several illustrative examples where existing metrics yield uninformative or
contradictory results. Furthermore, we analyze multiple design variants of
StyleGAN to better understand the relationships between the model architecture,
training methods, and the properties of the resulting sample distribution. In
the process, we identify new variants that improve the state-of-the-art. We
also perform the first principled analysis of truncation methods and identify
an improved method. Finally, we extend our metric to estimate the perceptual
quality of individual samples, and use this to study latent space
interpolations.

本研究提出了一种评估度量标准，以明确，非参数化表示真实和生成数据流形， 可以分别和可靠地测量图像生成任务中样本的质量和覆盖范围， 并且展示了该度量标准在 StyleGAN 和 BigGAN 方面的有效性。同时，我们分析了 StyleGAN 的多个设计变体以更好地了解模型架构，训练方法与样本分布属性之间的关系，并识别出新的改进方法。最后，我们将度量标准扩展到个体样本的感知质量估计，并使用它来研究潜空间插值。