Evaluating generative adversarial networks (GANs) is inherently challenging.
In this paper, we revisit several representative sample-based evaluation
metrics for GANs, and address the problem of how to evaluate the evaluation
metrics. We start with a few necessary conditions for metrics to produce
meaningful scores, such as distinguishing real from generated samples,
identifying mode dropping and mode collapsing, and detecting overfitting. With
a series of carefully designed experiments, we comprehensively investigate
existing sample-based metrics and identify their strengths and limitations in
practical settings. Based on these results, we observe that kernel Maximum Mean
Discrepancy (MMD) and the 1-Nearest-Neighbor (1-NN) two-sample test seem to
satisfy most of the desirable properties, provided that the distances between
samples are computed in a suitable feature space. Our experiments also unveil
interesting properties about the behavior of several popular GAN models, such
as whether they are memorizing training samples, and how far they are from
learning the target distribution.

本文研究了生成对抗网络（GAN）的评估方法，发现基于核最大均值差（MMD）和最近邻（1-NN）等指标，可以较好地满足评估需要，同时具备区分真实样本与生成样本的性质，并探究了几种著名 GAN 模型的特性。

生成对抗网络评估指标的实证研究

An empirical study on evaluation metrics of generative adversarial  networks

Generative adversarial networks (GANs) are pow- erful generative models based
on providing feed- back to a generative network via a discriminator network.
However, the discriminator usually as- sesses individual samples. This prevents
the dis- criminator from accessing global distributional statistics of
generated samples, and often leads to mode dropping: the generator models only
part of the target distribution. We propose to feed the discriminator with
mixed batches of true and fake samples, and train it to predict the ratio of
true samples in the batch. The latter score does not depend on the order of
samples in a batch. Rather than learning this invariance, we introduce a
generic permutation-invariant discriminator ar- chitecture. This architecture
is provably a uni- versal approximator of all symmetric functions.
Experimentally, our approach reduces mode col- lapse in GANs on two synthetic
datasets, and obtains good results on the CIFAR10 and CelebA datasets, both
qualitatively and quantitatively.

该研究提出了一种新的方法来减少 GANs 中的模式坍塌问题，即通过引入一种通用的置换不变鉴别器架构来训练一个预测混合批的真假样本比例的不变鉴别器得分来代替传统方式评估单个样本的鉴别器。实验证明，在两个合成数据集上，这种方法有效地减少了模式坍塌，而在 CIFAR10 和 CelebA 数据集上得到了良好的定性和定量结果。