This paper deals with the challenging task of synthesizing novel views for
in-the-wild photographs. Existing methods have shown promising results
leveraging monocular depth estimation and color inpainting with layered depth
representations. However, these methods still have limited capability to handle
scenes with complex 3D geometry. We propose a new method based on the
multiplane image (MPI) representation. To accommodate diverse scene layouts in
the wild and tackle the difficulty in producing high-dimensional MPI contents,
we design a network structure that consists of two novel modules, one for plane
depth adjustment and another for depth-aware color prediction. The former
adjusts the initial plane positions using the RGBD context feature and an
attention mechanism. Given adjusted depth values, the latter predicts the color
and density for each plane separately with proper inter-plane interactions
achieved via a feature masking strategy. To train our method, we construct
large-scale stereo training data using only unconstrained single-view image
collections by a simple yet effective warp-back strategy. The experiments on
both synthetic and real datasets demonstrate that our trained model works
remarkably well and achieves state-of-the-art results.

本文提出一种基于多平面图像（MPI）表示的新方法来合成野外照片的新视角，结合深度估计和颜色填充技术，通过多平面图像表示解决具有复杂 3D 几何的场景，使用大规模立体训练数据并经过实验验证达到最新成果。