Image representations, from SIFT and bag of visual words to Convolutional
Neural Networks (CNNs) are a crucial component of almost all computer vision
systems. However, our understanding of them remains limited. In this paper we
study several landmark representations, both shallow and deep, by a number of
complementary visualization techniques. These visualizations are based on the
concept of "natural pre-image", namely a natural-looking image whose
representation has some notable property. We study in particular three such
visualizations: inversion, in which the aim is to reconstruct an image from its
representation, activation maximization, in which we search for patterns that
maximally stimulate a representation component, and caricaturization, in which
the visual patterns that a representation detects in an image are exaggerated.
We pose these as a regularized energy-minimization framework and demonstrate
its generality and effectiveness. In particular, we show that this method can
invert representations such as HOG more accurately than recent alternatives
while being applicable to CNNs too. Among our findings, we show that several
layers in CNNs retain photographically accurate information about the image,
with different degrees of geometric and photometric invariance.

本文通过自然感知前像的概念，运用逆向重建，激活最大化和漫画创作等多种可视化技术来研究浅层和深度学习表示，提出了一种正则化能量最小化的框架，可以更准确地反演 HOG 表示，并有效应用于卷积神经网络，其中发现 CNN 的多个层可以保留关于图像的各种位置和光度变换的信息。