This paper proposes a novel framework to evaluate fluid simulation methods
based on crowd-sourced user studies in order to robustly gather large numbers
of opinions. The key idea for a robust and reliable evaluation is to use a
reference video from a carefully selected real-world setup in the user study.
By conducting a series of controlled user studies and comparing their
evaluation results, we observe various factors that affect the perceptual
evaluation. Our data show that the availability of a reference video makes the
evaluation consistent. We introduce this approach for computing scores of
simulation methods as visual accuracy metric. As an application of the proposed
framework, a variety of popular simulation methods are evaluated.

本文提出了一种全新的评估流体模拟方法的框架，基于众包式用户研究，以此来收集大量意见以获得精确可靠性，其核心思想是通过用户研究中精心挑选实际应用场景中的参考视频来进行评估，通过进行一系列的控制用户研究和比较他们的评估结果，揭示了影响感知评估的各种因素，数据表明参考视频的可用性使得评估结果更加一致，将此方法引入到计算视觉准确度指标的模拟方法评分中，并应用于对各种流行的模拟方法的评估。