Fr\'echet Video Distance (FVD), a prominent metric for evaluating video
generation models, is known to conflict with human perception occasionally. In
this paper, we aim to explore the extent of FVD's bias toward per-frame quality
over temporal realism and identify its sources. We first quantify the FVD's
sensitivity to the temporal axis by decoupling the frame and motion quality and
find that the FVD increases only slightly with large temporal corruption. We
then analyze the generated videos and show that via careful sampling from a
large set of generated videos that do not contain motions, one can drastically
decrease FVD without improving the temporal quality. Both studies suggest FVD's
bias towards the quality of individual frames. We further observe that the bias
can be attributed to the features extracted from a supervised video classifier
trained on the content-biased dataset. We show that FVD with features extracted
from the recent large-scale self-supervised video models is less biased toward
image quality. Finally, we revisit a few real-world examples to validate our
hypothesis.

探索 Fréchet Video Distance（一种评估视频生成模型的优秀指标）对每帧质量和时域真实度的偏向程度，并确定其来源；通过解耦帧质量和运动质量，量化 FVD 对时域轴的敏感性，发现 FVD 在大的时域损坏下略微增加；通过精心从不包含运动的大量生成视频中进行采样，显示能够显著减少 FVD 而不改善时域质量；研究表明 FVD 偏向于个别帧的质量；观察到偏向可以归因于从内容偏向数据集训练的受监督视频分类器提取的特征；结果显示使用最近的大规模自监督视频模型提取的特征的 FVD 对图像质量偏向较小；最后，通过验证我们的假设，重新审视了一些现实世界的例子。

关于 Fréchet 视频距离中的内容偏差

On the Content Bias in Fréchet Video Distance

Recent advances in deep generative models have lead to remarkable progress in
synthesizing high quality images. Following their successful application in
image processing and representation learning, an important next step is to
consider videos. Learning generative models of video is a much harder task,
requiring a model to capture the temporal dynamics of a scene, in addition to
the visual presentation of objects. While recent attempts at formulating
generative models of video have had some success, current progress is hampered
by (1) the lack of qualitative metrics that consider visual quality, temporal
coherence, and diversity of samples, and (2) the wide gap between purely
synthetic video data sets and challenging real-world data sets in terms of
complexity. To this extent we propose Fr\'{e}chet Video Distance (FVD), a new
metric for generative models of video, and StarCraft 2 Videos (SCV), a
benchmark of game play from custom starcraft 2 scenarios that challenge the
current capabilities of generative models of video. We contribute a large-scale
human study, which confirms that FVD correlates well with qualitative human
judgment of generated videos, and provide initial benchmark results on SCV.

近年来，深度生成模型在图像合成方面取得了可观的进展，但在视频领域的学习则更为困难，需要模型同时捕获场景的时间动态和对象的视觉呈现。该研究提出了用于评价视频生成模型的新评价度量 - Fréchet Video Distance，并提供了基于 StarCraft 2 的新的视频生成模型挑战基准进行了大规模的人体研究和初始基准结果。