Most Neural Radiance Fields (NeRFs) have poor generalization ability,
limiting their application when representing multiple scenes by a single model.
To ameliorate this problem, existing methods simply condition NeRF models on
image features, lacking the global understanding and modeling of the entire 3D
scene. Inspired by the significant success of mask-based modeling in other
research fields, we propose a masked ray and view modeling method for
generalizable NeRF (MRVM-NeRF), the first attempt to incorporate mask-based
pretraining into 3D implicit representations. Specifically, considering that
the core of NeRFs lies in modeling 3D representations along the rays and across
the views, we randomly mask a proportion of sampled points along the ray at
fine stage by discarding partial information obtained from multi-viewpoints,
targeting at predicting the corresponding features produced in the coarse
branch. In this way, the learned prior knowledge of 3D scenes during
pretraining helps the model generalize better to novel scenarios after
finetuning. Extensive experiments demonstrate the superiority of our proposed
MRVM-NeRF under various synthetic and real-world settings, both qualitatively
and quantitatively. Our empirical studies reveal the effectiveness of our
proposed innovative MRVM which is specifically designed for NeRF models.

提出了一种基于掩膜的光线和视角建模方法（MRVM-NeRF），通过细阶段上的随机掩膜来学习 3D 场景的先验知识，从而使模型的泛化能力得到提升。实验证明了该方法在合成和真实场景下具有优越性。