We present Worldsheet, a method for novel view synthesis using just a single RGB image as input. This is a challenging problem as it requires an understanding of the 3D geometry of the scene as well as texture mapping to generate both visible and occluded regions from new view-points. Our main insight is that simply shrink-wrapping a planar mesh sheet onto the input image, consistent with the learned intermediate depth, captures underlying geometry sufficient enough to generate photorealistic unseen views with arbitrarily large view-point changes. To operationalize this, we propose a novel differentiable texture sampler that allows our wrapped mesh sheet to be textured; which is then transformed into a target image via differentiable rendering. Our approach is category-agnostic, end-to-end trainable without using any 3D supervision and requires a single image at test time. Worldsheet consistently outperforms prior state-of-the-art methods on single-image view synthesis across several datasets. Furthermore, this simple idea captures novel views surprisingly well on a wide range of high resolution in-the-wild images in converting them into a navigable 3D pop-up. Video results and code at https://worldsheet.github.io

我们提出了一种名为Worldsheet的方法，可以使用单一的RGB图像进行新视角综合。方法利用可学习的中间深度一致地将平面网格板缩包到输入图像上，并掌握足以生成大视角变化下的逼真未见视角的底层几何知识。我们的方法是类不可知的，可在不使用任何3D监督的情况下进行端到端培训，并在测试时间只需要一张图像。我们还探讨了一种简单的扩展方式，通过堆叠多层Worldsheets来更好地处理遮挡。Worldsheet在几个数据集上持续超越先前的最先进方法。此外，这个简单的想法在各种高分辨率的现场照片上惊人地捕捉到了新视角，将它们转化为可导航的3D弹出窗口。

Worldsheet: 从单张图像生成视角综合的三维平面