We propose a method for depth estimation under different illumination
conditions, i.e., day and night time. As photometry is uninformative in regions
under low-illumination, we tackle the problem through a multi-sensor fusion
approach, where we take as input an additional synchronized sparse point cloud
(i.e., from a LiDAR) projected onto the image plane as a sparse depth map,
along with a camera image. The crux of our method lies in the use of the
abundantly available synthetic data to first approximate the 3D scene structure
by learning a mapping from sparse to (coarse) dense depth maps along with their
predictive uncertainty - we term this, SpaDe. In poorly illuminated regions
where photometric intensities do not afford the inference of local shape, the
coarse approximation of scene depth serves as a prior; the uncertainty map is
then used with the image to guide refinement through an uncertainty-driven
residual learning (URL) scheme. The resulting depth completion network
leverages complementary strengths from both modalities - depth is sparse but
insensitive to illumination and in metric scale, and image is dense but
sensitive with scale ambiguity. SpaDe can be used in a plug-and-play fashion,
which allows for 25% improvement when augmented onto existing methods to
preprocess sparse depth. We demonstrate URL on the nuScenes dataset where we
improve over all baselines by an average 11.65% in all-day scenarios, 11.23%
when tested specifically for daytime, and 13.12% for nighttime scenes.

我们提出了一种在不同光照条件下进行深度估计的方法，该方法通过多传感器融合的方式解决了低照度区域光度学信息不足的问题。我们的方法利用丰富的合成数据首先近似了三维场景结构，并通过学习从稀疏到（粗糙的）密集深度图的映射及其预测不确定性来引导深度的精修，从而提高了稀疏深度处理的效果。我们在 nuScenes 数据集上展示了该方法在不同场景中的改进表现。