Recently, intermediate feature maps of pre-trained convolutional neural
networks have shown significant perceptual quality improvements, when they are
used in the loss function for training new networks. It is believed that these
features are better at encoding the perceptual quality and provide more
efficient representations of input images compared to other perceptual metrics
such as SSIM and PSNR. However, there have been no systematic studies to
determine the underlying reason. Due to the lack of such an analysis, it is not
possible to evaluate the performance of a particular set of features or to
improve the perceptual quality even more by carefully selecting a subset of
features from a pre-trained CNN. This work shows that the capabilities of
pre-trained deep CNN features in optimizing the perceptual quality are
correlated with their success in capturing basic human visual perception
characteristics. In particular, we focus our analysis on fundamental aspects of
human perception, such as the contrast sensitivity and orientation selectivity.
We introduce two new formulations to measure the frequency and orientation
selectivity of the features learned by convolutional layers for evaluating deep
features learned by widely-used deep CNNs such as VGG-16. We demonstrate that
the pre-trained CNN features which receive higher scores are better at
predicting human quality judgment. Furthermore, we show the possibility of
using our method to select deep features to form a new loss function, which
improves the image reconstruction quality for the well-known single-image
super-resolution problem.

该研究分析了预训练卷积神经网络中的中间特征图如何提高感知质量，并提出了两种新的公式来评估深度 CNN 学习到的特征，发现使用这些特征能更好地预测人类质量判断， 进一步展示了利用该方法来选择深度特征用于形成新的损失函数可以提高单图超分辨率问题的图像重构质量。