Generating novel views from a single image remains a challenging task due to
the complexity of 3D scenes and the limited diversity in the existing
multi-view datasets to train a model on. Recent research combining large-scale
text-to-image (T2I) models with monocular depth estimation (MDE) has shown
promise in handling in-the-wild images. In these methods, an input view is
geometrically warped to novel views with estimated depth maps, then the warped
image is inpainted by T2I models. However, they struggle with noisy depth maps
and loss of semantic details when warping an input view to novel viewpoints. In
this paper, we propose a novel approach for single-shot novel view synthesis, a
semantic-preserving generative warping framework that enables T2I generative
models to learn where to warp and where to generate, through augmenting
cross-view attention with self-attention. Our approach addresses the
limitations of existing methods by conditioning the generative model on source
view images and incorporating geometric warping signals. Qualitative and
quantitative evaluations demonstrate that our model outperforms existing
methods in both in-domain and out-of-domain scenarios. Project page is
available at this https URL

提出了一种新颖的方法，通过将 T2I 生成模型与单目深度估计相结合的生成扭曲框架，使 T2I 模型能够学习何时扭曲和何时生成，从而解决了从单张图像合成新视角时存在的限制，并在领域内外的场景中优于现有方法。