An event camera is a novel vision sensor that can capture per-pixel
brightness changes and output a stream of asynchronous ``events''. It has
advantages over conventional cameras in those scenes with high-speed motions
and challenging lighting conditions because of the high temporal resolution,
high dynamic range, low bandwidth, low power consumption, and no motion blur.
Therefore, several supervised monocular depth estimation from events is
proposed to address scenes difficult for conventional cameras. However, depth
annotation is costly and time-consuming. In this paper, to lower the annotation
cost, we propose a self-supervised event-based monocular depth estimation
framework named EMoDepth. EMoDepth constrains the training process using the
cross-modal consistency from intensity frames that are aligned with events in
the pixel coordinate. Moreover, in inference, only events are used for
monocular depth prediction. Additionally, we design a multi-scale
skip-connection architecture to effectively fuse features for depth estimation
while maintaining high inference speed. Experiments on MVSEC and DSEC datasets
demonstrate that our contributions are effective and that the accuracy can
outperform existing supervised event-based and unsupervised frame-based
methods.

提出了一种自我监督的基于事件感知的单目深度估计框架 EMoDepth，利用来自与像素坐标中的事件对齐的强度帧的跨模态一致性约束训练过程，并且在推理中仅使用事件进行单目深度预测，同时设计了多尺度跳跃连接架构以有效融合用于深度估计的特征，并保持高推理速度。在 MVSEC 和 DSEC 数据集上的实验证明了方法的有效性，并且精度能够超过现有的有监督基于事件感知和无监督基于帧的方法。

基于自我监督的事件驱动单目深度估计与跨模态一致性

Self-supervised Event-based Monocular Depth Estimation using Cross-modal  Consistency

This paper aims to address the problem of supervised monocular depth
estimation. We start with a meticulous pilot study to demonstrate that the
long-range correlation is essential for accurate depth estimation. Therefore,
we propose to leverage the Transformer to model this global context with an
effective attention mechanism. We also adopt an additional convolution branch
to preserve the local information as the Transformer lacks the spatial
inductive bias in modeling such contents. However, independent branches lead to
a shortage of connections between features. To bridge this gap, we design a
hierarchical aggregation and heterogeneous interaction module to enhance the
Transformer features via element-wise interaction and model the affinity
between the Transformer and the CNN features in a set-to-set translation
manner. Due to the unbearable memory cost caused by global attention on
high-resolution feature maps, we introduce the deformable scheme to reduce the
complexity. Extensive experiments on the KITTI, NYU, and SUN RGB-D datasets
demonstrate that our proposed model, termed DepthFormer, surpasses
state-of-the-art monocular depth estimation methods with prominent margins.
Notably, it achieves the most competitive result on the highly competitive
KITTI depth estimation benchmark. Our codes and models are available at
this https URL

该研究提出了一种基于 Transformer 和 CNN 特征组合的分层聚合模型，以解决有监督单目深度估计的问题，实验结果表明，该模型在 KITTI、NYU 和 SUN RGB-D 等数据集上均超越了现有的有监督单目深度估计方法，特别是在 KITTI 深度估计基准测试上实现了最具竞争力的结果。