A good metric, which promises a reliable comparison between solutions, is essential to a well-defined task. Unlike most vision tasks that have per-sample ground-truth, image synthesis targets generating \emph{unseen} data and hence is usually evaluated with a distributional distance between one set of real samples and another set of generated samples. This work provides an empirical study on the evaluation of synthesis performance by taking the popular generative adversarial networks (GANs) as a representative of generative models. In particular, we make in-depth analyses on how to represent a data point in the feature space, how to calculate a fair distance using selected samples, and how many instances to use from each set. Experiments on multiple datasets and settings suggest that (1) a group of models including both CNN-based and ViT-based architectures serve as reliable and robust feature extractors, (2) Centered Kernel Alignment (CKA) enables better comparison across various extractors and hierarchical layers in one model, and (3) CKA shows satisfactory sample efficiency and complements existing metrics (\textit{e.g.}, FID) in characterizing the similarity between two internal data correlations. These findings help us design a new measurement system, based on which we re-evaluate the state-of-the-art generative models in a consistent and reliable way.

本文通过实证研究代表性生成模型GAN，深入分析了如何在特征空间中代表数据点、如何使用选定样本计算合理距离以及每组要使用多少个实例等问题。实验结果表明，基于CNN和ViT的架构都是可靠和稳健的特征提取器，CKA能够在一种模型中跨越各种提取器和分层层次上产生更好的比较，而且在表征两个内部数据相关性之间的相似性方面，CKA显示出令人满意的样本效率并补充了现有指标（例如FID）。这些发现帮助我们设计了一个新的测量系统，以便在一致可靠的方式下重新评估了最先进的生成模型。

重新审视使用GANs进行图像合成的评估