As with many machine learning problems, the progress of image generation
methods hinges on good evaluation metrics. One of the most popular is the
Frechet Inception Distance (FID). FID estimates the distance between a
distribution of Inception-v3 features of real images, and those of images
generated by the algorithm. We highlight important drawbacks of FID:
Inception's poor representation of the rich and varied content generated by
modern text-to-image models, incorrect normality assumptions, and poor sample
complexity. We call for a reevaluation of FID's use as the primary quality
metric for generated images. We empirically demonstrate that FID contradicts
human raters, it does not reflect gradual improvement of iterative
text-to-image models, it does not capture distortion levels, and that it
produces inconsistent results when varying the sample size. We also propose an
alternative new metric, CMMD, based on richer CLIP embeddings and the maximum
mean discrepancy distance with the Gaussian RBF kernel. It is an unbiased
estimator that does not make any assumptions on the probability distribution of
the embeddings and is sample efficient. Through extensive experiments and
analysis, we demonstrate that FID-based evaluations of text-to-image models may
be unreliable, and that CMMD offers a more robust and reliable assessment of
image quality.

FID 作为生成图像的主要质量度量标准存在一些问题，我们提出了一个基于更丰富的 CLIP 嵌入和最大均值差异距离的高斯 RBF 核的新的替代度量标准 CMMD，通过实验证明 FID 评估文本到图像模型的可靠性较低，而 CMMD 则提供了更强大和可靠的图像质量评估方法。