This paper shows that two commonly used evaluation metrics for generative models, the Fr\'echet Inception Distance (FID) and the Inception Score (IS), are biased -- the expected value of the score computed for a finite sample set is not the true value of the score. Worse, the paper shows that the bias term depends on the particular model being evaluated, so model A may get a better score than model B simply because model A's bias term is smaller. This effect cannot be fixed by evaluating at a fixed number of samples. This means all comparisons using FID or IS as currently computed are unreliable. We then show how to extrapolate the score to obtain an effectively bias-free estimate of scores computed with an infinite number of samples, which we term $\overline{\textrm{FID}}_\infty$ and $\overline{\textrm{IS}}_\infty$. In turn, this effectively bias-free estimate requires good estimates of scores with a finite number of samples. We show that using Quasi-Monte Carlo integration notably improves estimates of FID and IS for finite sample sets. Our extrapolated scores are simple, drop-in replacements for the finite sample scores. Additionally, we show that using low discrepancy sequence in GAN training offers small improvements in the resulting generator.

本文表明了当评估生成模型时，目前常用的两个评估指标 FID 和 IS 是存在偏差的，偏差的程度还与受评估的模型有关系；同时，本文提出了一种修正偏差的方法，即使用 Quasi-Monte Carlo 积分方法求得准确的评估值，使用这种方法可以获得更为准确的评估结果，并且该方法可直接取代现有的评估指标。

有效无偏的 FID 和 Inception Score 以及它们的获取方式