We propose an online multi-view depth prediction approach on posed video
streams, where the scene geometry information computed in the previous time
steps is propagated to the current time step in an efficient and geometrically
plausible way. The backbone of our approach is a real-time capable, lightweight
encoder-decoder that relies on cost volumes computed from pairs of images. We
extend it by placing a ConvLSTM cell at the bottleneck layer, which compresses
an arbitrary amount of past information in its states. The novelty lies in
propagating the hidden state of the cell by accounting for the viewpoint
changes between time steps. At a given time step, we warp the previous hidden
state into the current camera plane using the previous depth prediction. Our
extension brings only a small overhead of computation time and memory
consumption, while improving the depth predictions significantly. As a result,
we outperform the existing state-of-the-art multi-view stereo methods on most
of the evaluated metrics in hundreds of indoor scenes while maintaining a
real-time performance. Code available:
this https URL

该论文提出了一种在线多视角深度预测方法，使用 ConvLSTM cell 实现对过去一定量信息的压缩，并考虑了时间步之间的视点变化，通过对先前的深度预测进行位移实现细胞的隐藏态的传播。该方法在实时性能的基础上带来了显著的深度预测的提高，并在数百个室内场景中表现出了优异的状态，代码详见 https://thisURL。