Denoising diffusion probabilistic models have transformed image generation with their impressive fidelity and diversity. We show that they also excel in estimating optical flow and monocular depth, surprisingly, without task-specific architectures and loss functions that are predominant for these tasks. Compared to the point estimates of conventional regression-based methods, diffusion models also enable Monte Carlo inference, e.g., capturing uncertainty and ambiguity in flow and depth. With self-supervised pre-training, the combined use of synthetic and real data for supervised training, and technical innovations (infilling and step-unrolled denoising diffusion training) to handle noisy-incomplete training data, and a simple form of coarse-to-fine refinement, one can train state-of-the-art diffusion models for depth and optical flow estimation. Extensive experiments focus on quantitative performance against benchmarks, ablations, and the model's ability to capture uncertainty and multimodality, and impute missing values. Our model, DDVM (Denoising Diffusion Vision Model), obtains a state-of-the-art relative depth error of 0.074 on the indoor NYU benchmark and an Fl-all outlier rate of 3.26\% on the KITTI optical flow benchmark, about 25\% better than the best published method. For an overview see https://diffusion-vision.github.io.

本文展示了使用去噪扩散概率模型进行单目 depth 和 optical flow 估计，通过 Monte Carlo 推理，结合了自监督预训练和合成和真实数据的监督训练，能够在去噪和缺陷数据方面提供技术创新，以及对于不确定性和多模态的处理能力等方面都能取得最先进的效果。DDVM 是本文中提出的模型，相对深度误差为 0.074，比最佳发布方法的 Fl-all outlier rate 高 25%。

扩散模型在光流和单目深度估计中的惊人有效性