We consider the problem of reconstructing a dynamic scene observed from a
stereo camera. Most existing methods for depth from stereo treat different
stereo frames independently, leading to temporally inconsistent depth
predictions. Temporal consistency is especially important for immersive AR or
VR scenarios, where flickering greatly diminishes the user experience. We
propose DynamicStereo, a novel transformer-based architecture to estimate
disparity for stereo videos. The network learns to pool information from
neighboring frames to improve the temporal consistency of its predictions. Our
architecture is designed to process stereo videos efficiently through divided
attention layers. We also introduce Dynamic Replica, a new benchmark dataset
containing synthetic videos of people and animals in scanned environments,
which provides complementary training and evaluation data for dynamic stereo
closer to real applications than existing datasets. Training with this dataset
further improves the quality of predictions of our proposed DynamicStereo as
well as prior methods. Finally, it acts as a benchmark for consistent stereo
methods.

本文提出了一种新颖的基于 Transformer 的 DynamicStereo 结构来解决从双目摄像机中重构动态场景的问题，并介绍了 Dynamic Replica 数据集来提供更贴近实际应用的训练和评估数据。