Neural Radiance Field (NeRF) has achieved superior performance for novel view
synthesis by modeling the scene with a Multi-Layer Perception (MLP) and a
volume rendering procedure, however, when fewer known views are given (i.e.,
few-shot view synthesis), the model is prone to overfit the given views. To
handle this issue, previous efforts have been made towards leveraging learned
priors or introducing additional regularizations. In contrast, in this paper,
we for the first time provide an orthogonal method from the perspective of
network structure. Given the observation that trivially reducing the number of
model parameters alleviates the overfitting issue, but at the cost of missing
details, we propose the multi-input MLP (mi-MLP) that incorporates the inputs
(i.e., location and viewing direction) of the vanilla MLP into each layer to
prevent the overfitting issue without harming detailed synthesis. To further
reduce the artifacts, we propose to model colors and volume density separately
and present two regularization terms. Extensive experiments on multiple
datasets demonstrate that: 1) although the proposed mi-MLP is easy to
implement, it is surprisingly effective as it boosts the PSNR of the baseline
from $14.73$ to $24.23$. 2) the overall framework achieves state-of-the-art
results on a wide range of benchmarks. We will release the code upon
publication.

通过多输入多层感知器 (mi-MLP) 让神经光辐射场 (NeRF) 模型在少样本视角合成时避免过拟合问题，并提高 PSNR 值从 $14.73$ 提升到 $24.23$，并在多个基准测试中取得了最先进的结果。

神经辐射场中的 Vanilla MLP 对于少样本视图合成足够吗？

Is Vanilla MLP in Neural Radiance Field Enough for Few-shot View  Synthesis?

Novel view synthesis from limited observations remains an important and
persistent task. However, high efficiency in existing NeRF-based few-shot view
synthesis is often compromised to obtain an accurate 3D representation. To
address this challenge, we propose a few-shot view synthesis framework based on
3D Gaussian Splatting that enables real-time and photo-realistic view synthesis
with as few as three training views. The proposed method, dubbed FSGS, handles
the extremely sparse initialized SfM points with a thoughtfully designed
Gaussian Unpooling process. Our method iteratively distributes new Gaussians
around the most representative locations, subsequently infilling local details
in vacant areas. We also integrate a large-scale pre-trained monocular depth
estimator within the Gaussians optimization process, leveraging online
augmented views to guide the geometric optimization towards an optimal
solution. Starting from sparse points observed from limited input viewpoints,
our FSGS can accurately grow into unseen regions, comprehensively covering the
scene and boosting the rendering quality of novel views. Overall, FSGS achieves
state-of-the-art performance in both accuracy and rendering efficiency across
diverse datasets, including LLFF, Mip-NeRF360, and Blender. Project website:
this https URL

提出了基于 3D 高斯喷砂的少样本视角合成方法，通过高效地合成视图实现实时和照片般逼真的视角合成，方法称为 FSGS，通过设计精巧的高斯解卷积处理稀疏初始化的 SfM 点，通过分布新的高斯函数填补空白区域中的局部细节，在高斯优化过程中整合大规模预训练的单眼深度估计器，通过在线增强视图引导几何优化达到最优解，FSGS 在包括 LLFF、Mip-NeRF360 和 Blender 在内的各种数据集上实现了最先进的性能，包括准确性和渲染效率。