The visual system is hierarchically organized to process visual information
in successive stages. Neural representations vary drastically across the first
stages of visual processing: at the output of the retina, ganglion cell
receptive fields (RFs) exhibit a clear antagonistic center-surround structure,
whereas in the primary visual cortex, typical RFs are sharply tuned to a
precise orientation. There is currently no unified theory explaining these
differences in representations across layers. Here, using a deep convolutional
neural network trained on image recognition as a model of the visual system, we
show that such differences in representation can emerge as a direct consequence
of different neural resource constraints on the retinal and cortical networks,
and we find a single model from which both geometries spontaneously emerge at
the appropriate stages of visual processing. The key constraint is a reduced
number of neurons at the retinal output, consistent with the anatomy of the
optic nerve as a stringent bottleneck. Second, we find that, for simple
cortical networks, visual representations at the retinal output emerge as
nonlinear and lossy feature detectors, whereas they emerge as linear and
faithful encoders of the visual scene for more complex cortices. This result
predicts that the retinas of small vertebrates should perform sophisticated
nonlinear computations, extracting features directly relevant to behavior,
whereas retinas of large animals such as primates should mostly encode the
visual scene linearly and respond to a much broader range of stimuli. These
predictions could reconcile the two seemingly incompatible views of the retina
as either performing feature extraction or efficient coding of natural scenes,
by suggesting that all vertebrates lie on a spectrum between these two
objectives, depending on the degree of neural resources allocated to their
visual system.

通过使用深度卷积神经网络模型，本文发现视觉系统中的不同神经约束可以导致视觉处理的不同阶段产生不同的表征模式，并预测小型脊椎动物的视网膜应执行非线性的特征检测计算，而大型动物的视网膜应当基本以线性方式编码视觉场景以响应广泛的刺激。