We introduce MVSplat360, a feed-forward approach for 360{\deg} novel view synthesis (NVS) of diverse real-world scenes, using only sparse observations. This setting is inherently ill-posed due to minimal overlap among input views and insufficient visual information provided, making it challenging for conventional methods to achieve high-quality results. Our MVSplat360 addresses this by effectively combining geometry-aware 3D reconstruction with temporally consistent video generation. Specifically, it refactors a feed-forward 3D Gaussian Splatting (3DGS) model to render features directly into the latent space of a pre-trained Stable Video Diffusion (SVD) model, where these features then act as pose and visual cues to guide the denoising process and produce photorealistic 3D-consistent views. Our model is end-to-end trainable and supports rendering arbitrary views with as few as 5 sparse input views. To evaluate MVSplat360's performance, we introduce a new benchmark using the challenging DL3DV-10K dataset, where MVSplat360 achieves superior visual quality compared to state-of-the-art methods on wide-sweeping or even 360{\deg} NVS tasks. Experiments on the existing benchmark RealEstate10K also confirm the effectiveness of our model. The video results are available on our project page: https://donydchen.github.io/mvsplat360.

本研究解决了360度新视图合成中的稀疏观察问题，因输入视图之间重叠最小，导致传统方法难以获得高质量结果。MVSplat360通过结合几何感知的三维重建与时间一致的视频生成，提出了一种新颖的方法，显著提高了在仅使用5个稀疏输入视图时的合成质量。实验结果表明，MVSplat360在DL3DV-10K数据集上与现有最先进的方法相比，视觉质量显著提升。

MVSplat360：基于稀疏视图的前馈360场景合成