We present a method for novel view synthesis from input images that are
freely distributed around a scene. Our method does not rely on a regular
arrangement of input views, can synthesize images for free camera movement
through the scene, and works for general scenes with unconstrained geometric
layouts. We calibrate the input images via SfM and erect a coarse geometric
scaffold via MVS. This scaffold is used to create a proxy depth map for a novel
view of the scene. Based on this depth map, a recurrent encoder-decoder network
processes reprojected features from nearby views and synthesizes the new view.
Our network does not need to be optimized for a given scene. After training on
a dataset, it works in previously unseen environments with no fine-tuning or
per-scene optimization. We evaluate the presented approach on challenging
real-world datasets, including Tanks and Temples, where we demonstrate
successful view synthesis for the first time and substantially outperform prior
and concurrent work.

本文提出了一种新的方法，可以从自由分布在场景周围的输入图像中合成新视角的图像，不依赖于输入视角的规则排列，可以为场景中的自由相机运动合成图像，并适用于具有任意几何布局的通用场景，该方法通过 SfM 对输入图像进行校准并通过 MVS 建立一个粗略的几何支架，然后基于透视图创建一个代理深度图，进而利用循环编解码网络处理从附近视角重新投影的特征，并合成新视角。该网络不需要针对特定场景进行优化，通过数据集训练后，可以在之前看不到的环境中工作，本文的方法在 Tanks and Temples 等具有挑战性的现实世界数据集上进行了评估和实验证明，表现出色，大大优于之前和同时进行的工作。