One of the most impactful findings in computational neuroscience over the
past decade is that the object recognition accuracy of deep neural networks
(DNNs) correlates with their ability to predict neural responses to natural
images in the inferotemporal (IT) cortex. This discovery supported the
long-held theory that object recognition is a core objective of the visual
cortex, and suggested that more accurate DNNs would serve as better models of
IT neuron responses to images. Since then, deep learning has undergone a
revolution of scale: billion parameter-scale DNNs trained on billions of images
are rivaling or outperforming humans at visual tasks including object
recognition. Have today's DNNs become more accurate at predicting IT neuron
responses to images as they have grown more accurate at object recognition?
Surprisingly, across three independent experiments, we find this is not the
case. DNNs have become progressively worse models of IT as their accuracy has
increased on ImageNet. To understand why DNNs experience this trade-off and
evaluate if they are still an appropriate paradigm for modeling the visual
system, we turn to recordings of IT that capture spatially resolved maps of
neuronal activity elicited by natural images. These neuronal activity maps
reveal that DNNs trained on ImageNet learn to rely on different visual features
than those encoded by IT and that this problem worsens as their accuracy
increases. We successfully resolved this issue with the neural harmonizer, a
plug-and-play training routine for DNNs that aligns their learned
representations with humans. Our results suggest that harmonized DNNs break the
trade-off between ImageNet accuracy and neural prediction accuracy that assails
current DNNs and offer a path to more accurate models of biological vision.

本文发现，尽管深度学习神经网络在图像识别任务上比人类表现更好，但它们不能很好地预测自然图像在颞下皮层 (IT cortex) 的神经响应，我们提出了神经协调器，通过该工具可以纠正这个问题。