Depth estimation is crucial for interpreting complex environments, especially
in areas such as autonomous vehicle navigation and robotics. Nonetheless,
obtaining accurate depth readings from event camera data remains a formidable
challenge. Event cameras operate differently from traditional digital cameras,
continuously capturing data and generating asynchronous binary spikes that
encode time, location, and light intensity. Yet, the unique sampling mechanisms
of event cameras render standard image based algorithms inadequate for
processing spike data. This necessitates the development of innovative,
spike-aware algorithms tailored for event cameras, a task compounded by the
irregularity, continuity, noise, and spatial and temporal characteristics
inherent in spiking data.Harnessing the strong generalization capabilities of
transformer neural networks for spatiotemporal data, we propose a purely
spike-driven spike transformer network for depth estimation from spiking camera
data. To address performance limitations with Spiking Neural Networks (SNN), we
introduce a novel single-stage cross-modality knowledge transfer framework
leveraging knowledge from a large vision foundational model of artificial
neural networks (ANN) (DINOv2) to enhance the performance of SNNs with limited
data. Our experimental results on both synthetic and real datasets show
substantial improvements over existing models, with notable gains in Absolute
Relative and Square Relative errors (49% and 39.77% improvements over the
benchmark model Spike-T, respectively). Besides accuracy, the proposed model
also demonstrates reduced power consumptions, a critical factor for practical
applications.

基于脉冲相机数据的纯脉冲驱动脉冲变压器网络模型以及跨模态知识迁移框架，在深度估计领域取得了显著提升的结果，同时减少了功耗消耗。

基于跨模态知识蒸馏的事件相机深度估计的新型脉冲变压器网络

A Novel Spike Transformer Network for Depth Estimation from Event  Cameras via Cross-modality Knowledge Distillation

Recently, learning-based approaches show promising results in navigation
tasks. However, the poor generalization capability and the simulation-reality
gap prevent a wide range of applications. We consider the problem of improving
the generalization of mobile robots and achieving sim-to-real transfer for
navigation skills. To that end, we propose a cross-modal fusion method and a
knowledge transfer framework for better generalization. This is realized by a
teacher-student distillation architecture. The teacher learns a discriminative
representation and the near-perfect policy in an ideal environment. By
imitating the behavior and representation of the teacher, the student is able
to align the features from noisy multi-modal input and reduce the influence of
variations on navigation policy. We evaluate our method in simulated and
real-world environments. Experiments show that our method outperforms the
baselines by a large margin and achieves robust navigation performance with
varying working conditions.

通过跨模态融合方法和知识迁移框架，提出了一种改善移动机器人的泛化能力并实现从模拟到实际场景的导航技能转移的方法，通过教师 - 学生蒸馏架构，在理想环境中学习判别性表示和接近完美的策略，通过模仿教师的行为和表示，学生能够对来自多噪声模态输入的特征进行对齐，并减小变异对导航策略的影响，实验结果表明我们的方法在模拟和实际环境中全面超越了基准方法，并在不同工作条件下实现了稳健的导航性能。