Color-guided depth super-resolution (DSR) is an encouraging paradigm that
enhances a low-resolution (LR) depth map guided by an extra high-resolution
(HR) RGB image from the same scene. Existing methods usually use interpolation
to upscale the depth maps before feeding them into the network and transfer the
high-frequency information extracted from HR RGB images to guide the
reconstruction of depth maps. However, the extracted high-frequency information
usually contains textures that are not present in depth maps in the existence
of the cross-modality gap, and the noises would be further aggravated by
interpolation due to the resolution gap between the RGB and depth images. To
tackle these challenges, we propose a novel Symmetric Uncertainty-aware Feature
Transmission (SUFT) for color-guided DSR. (1) For the resolution gap, SUFT
builds an iterative up-and-down sampling pipeline, which makes depth features
and RGB features spatially consistent while suppressing noise amplification and
blurring by replacing common interpolated pre-upsampling. (2) For the
cross-modality gap, we propose a novel Symmetric Uncertainty scheme to remove
parts of RGB information harmful to the recovery of HR depth maps. Extensive
experiments on benchmark datasets and challenging real-world settings suggest
that our method achieves superior performance compared to state-of-the-art
methods. Our code and models are available at
this https URL

本文提出了一种新颖的对称不确定性特征传输（SUFT）来解决深度图超分辨率中颜色指导的挑战，包括分辨率差异和跨模态差异，并取得了优于现有方法的表现。

深度超分辨率对称不确定性感知特征传输

Symmetric Uncertainty-Aware Feature Transmission for Depth  Super-Resolution

Depth completion involves recovering a dense depth map from a sparse map and
an RGB image. Recent approaches focus on utilizing color images as guidance
images to recover depth at invalid pixels. However, color images alone are not
enough to provide the necessary semantic understanding of the scene.
Consequently, the depth completion task suffers from sudden illumination
changes in RGB images (e.g., shadows). In this paper, we propose a novel
three-branch backbone comprising color-guided, semantic-guided, and
depth-guided branches. Specifically, the color-guided branch takes a sparse
depth map and RGB image as an input and generates color depth which includes
color cues (e.g., object boundaries) of the scene. The predicted dense depth
map of color-guided branch along-with semantic image and sparse depth map is
passed as input to semantic-guided branch for estimating semantic depth. The
depth-guided branch takes sparse, color, and semantic depths to generate the
dense depth map. The color depth, semantic depth, and guided depth are
adaptively fused to produce the output of our proposed three-branch backbone.
In addition, we also propose to apply semantic-aware multi-modal
attention-based fusion block (SAMMAFB) to fuse features between all three
branches. We further use CSPN++ with Atrous convolutions to refine the dense
depth map produced by our three-branch backbone. Extensive experiments show
that our model achieves state-of-the-art performance in the KITTI depth
completion benchmark at the time of submission.

本文提出一种新的三支架构解决深度补全问题，其中包括颜色导向的、语义导向的和深度导向的分支；同时使用基于多模态注意力融合的方法来融合三支架构，从 RGB 图像、稀疏深度图和语义图中预测出密集深度图。经过广泛的实验证明，在 KITTI 深度补全基准测试中取得了最先进的性能。

SemAttNet: 基于注意力的语义感知引导深度完成

SemAttNet: Towards Attention-based Semantic Aware Guided Depth  Completion

Depth map super-resolution is a task with high practical application
requirements in the industry. Existing color-guided depth map super-resolution
methods usually necessitate an extra branch to extract high-frequency detail
information from RGB image to guide the low-resolution depth map
reconstruction. However, because there are still some differences between the
two modalities, direct information transmission in the feature dimension or
edge map dimension cannot achieve satisfactory result, and may even trigger
texture copying in areas where the structures of the RGB-D pair are
inconsistent. Inspired by the multi-task learning, we propose a joint learning
network of depth map super-resolution (DSR) and monocular depth estimation
(MDE) without introducing additional supervision labels. For the interaction of
two subnetworks, we adopt a differentiated guidance strategy and design two
bridges correspondingly. One is the high-frequency attention bridge (HABdg)
designed for the feature encoding process, which learns the high-frequency
information of the MDE task to guide the DSR task. The other is the content
guidance bridge (CGBdg) designed for the depth map reconstruction process,
which provides the content guidance learned from DSR task for MDE task. The
entire network architecture is highly portable and can provide a paradigm for
associating the DSR and MDE tasks. Extensive experiments on benchmark datasets
demonstrate that our method achieves competitive performance. Our code and
models are available at this https URL

本文提出一个基于多任务学习的联合深度图超分辨率与单目深度估计网络，采用不同的引导策略设计两个桥梁，一个是用于特征编码过程的高频注意力桥梁，另一个是用于深度图重建过程的内容引导桥梁，实验证明该方法在基准数据集上具有竞争性能。