Synthesizing a novel view from a single input image is a challenging task. Traditionally, this task was approached by estimating scene depth, warping, and inpainting, with machine learning models enabling parts of the pipeline. More recently, generative models are being increasingly employed in novel view synthesis (NVS), often encompassing the entire end-to-end system. In this work, we adapt a modern diffusion model architecture for end-to-end NVS in the pixel space, substantially outperforming previous state-of-the-art (SOTA) techniques. We explore different ways to encode geometric information into the network. Our experiments show that while these methods may enhance performance, their impact is minor compared to utilizing improved generative models. Moreover, we introduce a novel NVS training scheme that utilizes single-view datasets, capitalizing on their relative abundance compared to their multi-view counterparts. This leads to improved generalization capabilities to scenes with out-of-domain content.

本文解决了从单个输入图像合成新视图这一具有挑战性的任务。我们采用现代扩散模型架构进行端到端的视图合成，显著超越了之前的最先进技术。研究表明，尽管几何信息编码方法可能提升性能，但与改进的生成模型相比，其影响较小，并且引入的新训练方案利用单视图数据集，提升了对非领域内容场景的泛化能力。

基于像素空间扩散模型的新视图合成