Single image view synthesis allows for the generation of new views of a scene
given a single input image. This is challenging, as it requires comprehensively
understanding the 3D scene from a single image. As a result, current methods
typically use multiple images, train on ground-truth depth, or are limited to
synthetic data. We propose a novel end-to-end model for this task; it is
trained on real images without any ground-truth 3D information. To this end, we
introduce a novel differentiable point cloud renderer that is used to transform
a latent 3D point cloud of features into the target view. The projected
features are decoded by our refinement network to inpaint missing regions and
generate a realistic output image. The 3D component inside of our generative
model allows for interpretable manipulation of the latent feature space at test
time, e.g. we can animate trajectories from a single image. Unlike prior work,
we can generate high resolution images and generalise to other input
resolutions. We outperform baselines and prior work on the Matterport, Replica,
and RealEstate10K datasets.

本文提出了一种使用真实图像来训练、无需 3D 场景真值信息，通过可微分点云渲染器将潜在 3D 特征点云转换为目标视图输出图像，并通过细化网络解码来填补缺失区域的新型端到端模型，在测试时可以对潜在特征空间进行可解释的操作，可以生成高分辨率图像并推广到其他输入分辨率，将在 Matterport、Replica 和 RealEstate10K 数据集上优于基线和之前的工作。