As a bio-inspired vision sensor, the spike camera emulates the operational
principles of the fovea, a compact retinal region, by employing spike
discharges to encode the accumulation of per-pixel luminance intensity.
Leveraging its high temporal resolution and bio-inspired neuromorphic design,
the spike camera holds significant promise for advancing computer vision
applications. Saliency detection mimics the behavior of human beings and
captures the most salient region from the scenes. In this paper, we investigate
the visual saliency in the continuous spike stream for the first time. To
effectively process the binary spike stream, we propose a Recurrent Spiking
Transformer (RST) framework, which is based on a full spiking neural network.
Our framework enables the extraction of spatio-temporal features from the
continuous spatio-temporal spike stream while maintaining low power
consumption. To facilitate the training and validation of our proposed model,
we build a comprehensive real-world spike-based visual saliency dataset,
enriched with numerous light conditions. Extensive experiments demonstrate the
superior performance of our Recurrent Spiking Transformer framework in
comparison to other spike neural network-based methods. Our framework exhibits
a substantial margin of improvement in capturing and highlighting visual
saliency in the spike stream, which not only provides a new perspective for
spike-based saliency segmentation but also shows a new paradigm for full
SNN-based transformer models. The code and dataset are available at
https://github.com/BIT-Vision/SVS.

我们研究了连续脉冲流中的视觉显著性，并提出了一种基于完整脉冲神经网络的循环脉冲转换器框架，通过从连续的时空脉冲流中提取时空特征来实现对二进制脉冲流的有效处理。实验结果表明，我们的循环脉冲转换器框架在捕捉和突出脉冲流中的视觉显著性方面表现出了明显的改进幅度，不仅为基于脉冲的显著性分割提供了新的视角，也展示了全脉冲神经网络的转换器模型的新范式。

批处理视觉突发流中的视觉吸引力

Finding Visual Saliency in Continuous Spike Stream

Most monocular depth sensing methods use conventionally captured images that
are created without considering scene content. In contrast, animal eyes have
fast mechanical motions, called saccades, that control how the scene is imaged
by the fovea, where resolution is highest. In this paper, we present the
SaccadeCam framework for adaptively distributing resolution onto regions of
interest in the scene. Our algorithm for adaptive resolution is a
self-supervised network and we demonstrate results for end-to-end learning for
monocular depth estimation. We also show preliminary results with a real
SaccadeCam hardware prototype.

本文提出了一种名为 SaccadeCam 的方法，通过自适应地分配分辨率到场景中的兴趣区域，实现单目深度估计的端到端学习。该算法将充分考虑物体在视觉系统中的感知特性。

SaccadeCam：自适应的视觉注意力在单目深度感知中的应用

SaccadeCam: Adaptive Visual Attention for Monocular Depth Sensing

We develop a sampling extension of M-theory focused on invariance to scale
and translation. Quite surprisingly, the theory predicts an architecture of
early vision with increasing receptive field sizes and a high resolution fovea
-- in agreement with data about the cortical magnification factor, V1 and the
retina. From the slope of the inverse of the magnification factor, M-theory
predicts a cortical "fovea" in V1 in the order of $40$ by $40$ basic units at
each receptive field size -- corresponding to a foveola of size around $26$
minutes of arc at the highest resolution, $\approx 6$ degrees at the lowest
resolution. It also predicts uniform scale invariance over a fixed range of
scales independently of eccentricity, while translation invariance should
depend linearly on spatial frequency. Bouma's law of crowding follows in the
theory as an effect of cortical area-by-cortical area pooling; the Bouma
constant is the value expected if the signature responsible for recognition in
the crowding experiments originates in V2. From a broader perspective, the
emerging picture suggests that visual recognition under natural conditions
takes place by composing information from a set of fixations, with each
fixation providing recognition from a space-scale image fragment -- that is an
image patch represented at a set of increasing sizes and decreasing
resolutions.

本文提出了基于尺度和平移不变性的 M 理论采样扩展，并预测了越来越大的感受野大小和高分辨率视点的早期视觉架构。该论文认为视觉识别发生在一组凝视点上，每个凝视点提供一个空间尺度的图像片段的识别。