We introduce an approach to enhance the novel view synthesis from images
taken from a freely moving camera. The introduced approach focuses on outdoor
scenes where recovering accurate geometric scaffold and camera pose is
challenging, leading to inferior results using the state-of-the-art stable view
synthesis (SVS) method. SVS and related methods fail for outdoor scenes
primarily due to (i) over-relying on the multiview stereo (MVS) for geometric
scaffold recovery and (ii) assuming COLMAP computed camera poses as the best
possible estimates, despite it being well-studied that MVS 3D reconstruction
accuracy is limited to scene disparity and camera-pose accuracy is sensitive to
key-point correspondence selection. This work proposes a principled way to
enhance novel view synthesis solutions drawing inspiration from the basics of
multiple view geometry. By leveraging the complementary behavior of MVS and
monocular depth, we arrive at a better scene depth per view for nearby and far
points, respectively. Moreover, our approach jointly refines camera poses with
image-based rendering via multiple rotation averaging graph optimization. The
recovered scene depth and the camera-pose help better view-dependent on-surface
feature aggregation of the entire scene. Extensive evaluation of our approach
on the popular benchmark dataset, such as Tanks and Temples, shows substantial
improvement in view synthesis results compared to the prior art. For instance,
our method shows 1.5 dB of PSNR improvement on the Tank and Temples. Similar
statistics are observed when tested on other benchmark datasets such as FVS,
Mip-NeRF 360, and DTU.

本文介绍了一种基于多视角几何基础的方法，旨在提高自由移动相机拍摄图像的新观点综合效果。该方法利用多视角立体匹配和单目深度信息的互补特性以及基于多重旋转平均图优化来联合优化场景深度和相机位姿，并综合聚合整个场景的特征以提高视景综合效果。本文丰富的评估表明，该方法较之现有技术提高了相当多的综合效果。