Integrating an RGB camera into a ToF imaging system has become a significant
technique for perceiving the real world. The RGB guided ToF imaging system is
crucial to several applications, including face anti-spoofing, saliency
detection, and trajectory prediction. Depending on the distance of the working
range, the implementation schemes of the RGB guided ToF imaging systems are
different. Specifically, ToF sensors with a uniform field of illumination,
which can output dense depth but have low resolution, are typically used for
close-range measurements. In contrast, LiDARs, which emit laser pulses and can
only capture sparse depth, are usually employed for long-range detection. In
the two cases, depth quality improvement for RGB guided ToF imaging corresponds
to two sub-tasks: guided depth super-resolution and guided depth completion. In
light of the recent significant boost to the field provided by deep learning,
this paper comprehensively reviews the works related to RGB guided ToF imaging,
including network structures, learning strategies, evaluation metrics,
benchmark datasets, and objective functions. Besides, we present quantitative
comparisons of state-of-the-art methods on widely used benchmark datasets.
Finally, we discuss future trends and the challenges in real applications for
further research.

本文全面回顾了与 RGB 引导的 ToF 图像相关的工作，包括网络结构、学习策略、评估指标、基准数据集和目标函数，并在广泛使用的基准数据集上对最先进的方法进行定量比较。最后，我们讨论了未来的趋势和进一步研究中的实际应用挑战。