The human visual system uses two parallel pathways for spatial processing and
object recognition. In contrast, computer vision systems tend to use a single
feedforward pathway, rendering them less robust, adaptive, or efficient than
human vision. To bridge this gap, we developed a dual-stream vision model
inspired by the human eyes and brain. At the input level, the model samples two
complementary visual patterns to mimic how the human eyes use magnocellular and
parvocellular retinal ganglion cells to separate retinal inputs to the brain.
At the backend, the model processes the separate input patterns through two
branches of convolutional neural networks (CNN) to mimic how the human brain
uses the dorsal and ventral cortical pathways for parallel visual processing.
The first branch (WhereCNN) samples a global view to learn spatial attention
and control eye movements. The second branch (WhatCNN) samples a local view to
represent the object around the fixation. Over time, the two branches interact
recurrently to build a scene representation from moving fixations. We compared
this model with the human brains processing the same movie and evaluated their
functional alignment by linear transformation. The WhereCNN and WhatCNN
branches were found to differentially match the dorsal and ventral pathways of
the visual cortex, respectively, primarily due to their different learning
objectives. These model-based results lead us to speculate that the distinct
responses and representations of the ventral and dorsal streams are more
influenced by their distinct goals in visual attention and object recognition
than by their specific bias or selectivity in retinal inputs. This dual-stream
model takes a further step in brain-inspired computer vision, enabling parallel
neural networks to actively explore and understand the visual surroundings.

人类视觉系统使用两个并行通路进行空间处理和物体识别，而计算机视觉系统往往使用单一的前馈通路，导致其鲁棒性、适应性和效率低于人类视觉。为了弥补这一差距，我们开发了一种受人类眼睛和大脑启发的双流视觉模型，模拟人眼使用巨细胞和小细胞视网膜神经节细胞将视网膜输入分离到大脑的过程。通过对比人脑和模型处理同一视频的功能对齐性，我们发现 WhereCNN 和 WhatCNN 分支分别与视觉皮层的背侧和腹侧通路相匹配，主要因为它们的不同学习目标在视觉注意和物体识别中的显著差异。这种双流模型在脑启发的计算机视觉中迈出了进一步的步伐，使并行神经网络能够主动探索和理解视觉环境。