High frame rate and accurate depth estimation plays an important role in several tasks crucial to robotics and automotive perception. To date, this can be achieved through ToF and LiDAR devices for indoor and outdoor applications, respectively. However, their applicability is limited by low frame rate, energy consumption, and spatial sparsity. Depth on Demand (DoD) allows for accurate temporal and spatial depth densification achieved by exploiting a high frame rate RGB sensor coupled with a potentially lower frame rate and sparse active depth sensor. Our proposal jointly enables lower energy consumption and denser shape reconstruction, by significantly reducing the streaming requirements on the depth sensor thanks to its three core stages: i) multi-modal encoding, ii) iterative multi-modal integration, and iii) depth decoding. We present extended evidence assessing the effectiveness of DoD on indoor and outdoor video datasets, covering both environment scanning and automotive perception use cases.

本研究解决了低帧率传感器在机器人和汽车感知中深度估计精度不足的问题。提出的按需深度(DoD)方法，通过结合高帧率RGB传感器和低帧率稀疏深度传感器，实现了能耗降低和高密度形状重建。该方法在室内外视频数据集上的有效性评估表明，其在环境扫描和汽车感知的应用潜力显著。

按需深度：从低帧率主动传感器流式传输密集深度