Event cameras can record scene dynamics with high temporal resolution,
providing rich scene details for monocular depth estimation (MDE) even at
low-level illumination. Therefore, existing complementary learning approaches
for MDE fuse intensity information from images and scene details from event
data for better scene understanding. However, most methods directly fuse two
modalities at pixel level, ignoring that the attractive complementarity mainly
impacts high-level patterns that only occupy a few pixels. For example, event
data is likely to complement contours of scene objects. In this paper, we
discretize the scene into a set of high-level patterns to explore the
complementarity and propose a Pattern-based Complementary learning architecture
for monocular Depth estimation (PCDepth). Concretely, PCDepth comprises two
primary components: a complementary visual representation learning module for
discretizing the scene into high-level patterns and integrating complementary
patterns across modalities and a refined depth estimator aimed at scene
reconstruction and depth prediction while maintaining an efficiency-accuracy
balance. Through pattern-based complementary learning, PCDepth fully exploits
two modalities and achieves more accurate predictions than existing methods,
especially in challenging nighttime scenarios. Extensive experiments on MVSEC
and DSEC datasets verify the effectiveness and superiority of our PCDepth.
Remarkably, compared with state-of-the-art, PCDepth achieves a 37.9%
improvement in accuracy in MVSEC nighttime scenarios.

通过模式化的互补学习，PCDepth 充分利用两种模式，在夜间场景中获得比现有方法更精确的预测，MVSEC 夜间场景中准确率提高了 37.9%。

PCDepth: 基于模式的单目深度估计的互补学习方法

PCDepth: Pattern-based Complementary Learning for Monocular Depth  Estimation by Best of Both Worlds

Depth completion is the task of recovering dense depth maps from sparse ones,
usually with the help of color images. Existing image-guided methods perform
well on daytime depth perception self-driving benchmarks, but struggle in
nighttime scenarios with poor visibility and complex illumination. To address
these challenges, we propose a simple yet effective framework called LDCNet.
Our key idea is to use Recurrent Inter-Convolution Differencing (RICD) and
Illumination-Affinitive Intra-Convolution Differencing (IAICD) to enhance the
nighttime color images and reduce the negative effects of the varying
illumination, respectively. RICD explicitly estimates global illumination by
differencing two convolutions with different kernels, treating the
small-kernel-convolution feature as the center of the large-kernel-convolution
feature in a new perspective. IAICD softly alleviates local relative light
intensity by differencing a single convolution, where the center is dynamically
aggregated based on neighboring pixels and the estimated illumination map in
RICD. On both nighttime depth completion and depth estimation tasks, extensive
experiments demonstrate the effectiveness of our LDCNet, reaching the state of
the art.

本文提出了一种名为 LDCNet 的简单而有效的框架，使用先有的图像数据增强技术解决了夜间深度感知的困难，包括使用循环互卷积差异（RICD）和内部卷积差异（IAICD）等方法对夜间图像进行增强，并在深度完成和深度估计任务上取得了最新的成果。

夜间深度感知的可学习分差中心

Learnable Differencing Center for Nighttime Depth Perception

Monocular depth estimation aims at predicting depth from a single image or
video. Recently, self-supervised methods draw much attention since they are
free of depth annotations and achieve impressive performance on several daytime
benchmarks. However, they produce weird outputs in more challenging nighttime
scenarios because of low visibility and varying illuminations, which bring weak
textures and break brightness-consistency assumption, respectively. To address
these problems, in this paper we propose a novel framework with several
improvements: (1) we introduce Priors-Based Regularization to learn
distribution knowledge from unpaired depth maps and prevent model from being
incorrectly trained; (2) we leverage Mapping-Consistent Image Enhancement
module to enhance image visibility and contrast while maintaining brightness
consistency; and (3) we present Statistics-Based Mask strategy to tune the
number of removed pixels within textureless regions, using dynamic statistics.
Experimental results demonstrate the effectiveness of each component.
Meanwhile, our framework achieves remarkable improvements and state-of-the-art
results on two nighttime datasets.

本文提出了一种新的单目深度估计框架，引入了基于先验的正则化来学习配准深度图像的分布知识以避免错误训练，并使用 Mapping-Consistent Image Enhancement module 来增强图像可视性和对比度，同时维持亮度一致性，最后使用基于统计的掩码策略调整纹理区域内删除像素数量的数量来解决在夜间场景下的问题，实验表明每个部分都是有效的，并取得了令人瞩目的成果和最新成果。