Distinct scientific theories can make similar predictions. To adjudicate between theories, we must design experiments for which the theories make distinct predictions. Here we consider the problem of comparing deep neural networks as models of human visual recognition. To efficiently determine which models better explain human responses, we synthesize controversial stimuli: images for which different models produce distinct responses. We tested nine different models, which employed different architectures and recognition algorithms, including discriminative and generative models, all trained to recognize handwritten digits (from the MNIST set of digit images). We synthesized controversial stimuli to maximize the disagreement among the models. Human subjects viewed hundreds of these stimuli and judged the probability of presence of each digit in each image. We quantified how accurately each model predicted the human judgements. We found that the generative models (which learn the distribution of images for each class) better predicted the human judgments than the discriminative models (which learn to directly map from images to labels). The best performing model was the generative Analysis-by-Synthesis model (based on variational autoencoders). However, a simpler generative model (based on Gaussian-kernel-density estimation) also performed better than each of the discriminative models. None of the candidate models fully explained the human responses. We discuss the advantages and limitations of controversial stimuli as an experimental paradigm and how they generalize and improve on adversarial examples as probes of discrepancies between models and human perception.

为了比较深度神经网络作为人类视觉识别模型的能力，研究者们开发出了一种使用争议性刺激的方法，这些刺激让不同架构和算法的模型产生不同的响应，在手写数字和小自然图像识别任务中，最佳表现的模型是基于变分自编码器的生成分析合成模型和基于混合鉴别生成的联合能量模型，这些模型能更好地模拟图像分布和人类视觉，但没有一个模型能完全解释人的反应，争议性刺激提供了一种强有力的方法来检查模型与人类感知之间的差异。

争议性刺激：将神经网络之间相互对抗作为人类识别模型