The visual pathway of human brain includes two sub-pathways, ie, the ventral
pathway and the dorsal pathway, which focus on object identification and
dynamic information modeling, respectively. Both pathways comprise multi-layer
structures, with each layer responsible for processing different aspects of
visual information. Inspired by visual information processing mechanism of the
human brain, we propose the Brain Inspired Masked Modeling (BIMM) framework,
aiming to learn comprehensive representations from videos. Specifically, our
approach consists of ventral and dorsal branches, which learn image and video
representations, respectively. Both branches employ the Vision Transformer
(ViT) as their backbone and are trained using masked modeling method. To
achieve the goals of different visual cortices in the brain, we segment the
encoder of each branch into three intermediate blocks and reconstruct
progressive prediction targets with light weight decoders. Furthermore, drawing
inspiration from the information-sharing mechanism in the visual pathways, we
propose a partial parameter sharing strategy between the branches during
training. Extensive experiments demonstrate that BIMM achieves superior
performance compared to the state-of-the-art methods.

我们提出了脑启发的 Masked 建模 (BIMM) 框架，通过学习图像和视频表示，使用视觉变换器 (ViT) 作为骨干，并在训练过程中采用部分参数共享策略，在模拟人脑的视觉通路中取得了优越的性能。

BIMM: 基于脑神经灵感的视频表示学习

BIMM: Brain Inspired Masked Modeling for Video Representation Learning

The visual pathway involves complex networks of cells and regions which
contribute to the encoding and processing of visual information. While some
aspects of visual perception are understood, there are still many unanswered
questions regarding the exact mechanisms of visual encoding and the
organization of visual information along the pathway. This chapter discusses
the importance of visual perception and the challenges associated with
understanding how visual information is encoded and represented in the brain.
Furthermore, this chapter introduces the concept of neuroprostheses: devices
designed to enhance or replace bodily functions, and highlights the importance
of constructing computational models of the visual pathway in the
implementation of such devices. A number of such models, employing the use of
deep learning models, are outlined, and their value to understanding visual
coding and natural vision is discussed.

视觉途径、视觉感知、视觉编码、神经假肢和计算模型是重要研究领域。该文章介绍了视觉感知的重要性，讨论了视觉信息编码和大脑中信息组织的机制，以及如何使用计算模型来实现神经假肢并提高对于视觉编码和自然视觉的理解。

深度学习用于视觉神经假肢

Deep Learning for Visual Neuroprosthesis

We present a simple mathematical model for the mammalian low visual pathway,
taking into account its key elements: retina, lateral geniculate nucleus (LGN),
primary visual cortex (V1). The analogies between the cortical level of the
visual system and the structure of popular CNNs, used in image classification
tasks, suggests the introduction of an additional preliminary convolutional
module inspired to precortical neuronal circuits to improve robustness with
respect to global light intensity and contrast variations in the input images.
We validate our hypothesis on the popular databases MNIST, FashionMNIST and
SVHN, obtaining significantly more robust CNNs with respect to these
variations, once such extra module is added.

研究发现，在哺乳动物的低级视觉通路中，加入一种基于前皮层神经元回路的卷积模块可以显著提高卷积神经网络的鲁棒性。