3D reconstruction from a single image is a long-standing problem in computer vision. Learning-based methods address its inherent scale ambiguity by leveraging increasingly large labeled and unlabeled datasets, to produce geometric priors capable of generating accurate predictions across domains. As a result, state of the art approaches show impressive performance in zero-shot relative and metric depth estimation. Recently, diffusion models have exhibited remarkable scalability and generalizable properties in their learned representations. However, because these models repurpose tools originally designed for image generation, they can only operate on dense ground-truth, which is not available for most depth labels, especially in real-world settings. In this paper we present GRIN, an efficient diffusion model designed to ingest sparse unstructured training data. We use image features with 3D geometric positional encodings to condition the diffusion process both globally and locally, generating depth predictions at a pixel-level. With comprehensive experiments across eight indoor and outdoor datasets, we show that GRIN establishes a new state of the art in zero-shot metric monocular depth estimation even when trained from scratch.

本研究解决了从单幅图像进行3D重建时存在的尺度模糊问题，提出了一种名为GRIN的高效扩散模型，能够处理稀疏无结构的训练数据。通过在扩散过程中结合图像特征与3D几何位置编码，该方法在跨八个室内外数据集实验中展示了新的零-shot标准单目深度估计的最佳性能，具有重要的潜在应用价值。

GRIN：基于像素级扩散的零-shot度量深度估计