NeRF (Neural Radiance Fields) has demonstrated tremendous potential in novel
view synthesis and 3D reconstruction, but its performance is sensitive to input
image quality, which struggles to achieve high-fidelity rendering when provided
with low-quality sparse input viewpoints. Previous methods for NeRF restoration
are tailored for specific degradation type, ignoring the generality of
restoration. To overcome this limitation, we propose a generic radiance fields
restoration pipeline, named RaFE, which applies to various types of
degradations, such as low resolution, blurriness, noise, compression artifacts,
or their combinations. Our approach leverages the success of off-the-shelf 2D
restoration methods to recover the multi-view images individually. Instead of
reconstructing a blurred NeRF by averaging inconsistencies, we introduce a
novel approach using Generative Adversarial Networks (GANs) for NeRF generation
to better accommodate the geometric and appearance inconsistencies present in
the multi-view images. Specifically, we adopt a two-level tri-plane
architecture, where the coarse level remains fixed to represent the low-quality
NeRF, and a fine-level residual tri-plane to be added to the coarse level is
modeled as a distribution with GAN to capture potential variations in
restoration. We validate RaFE on both synthetic and real cases for various
restoration tasks, demonstrating superior performance in both quantitative and
qualitative evaluations, surpassing other 3D restoration methods specific to
single task. Please see our project website
this https URL

我们提出了一种名为 RaFE 的通用辐射场修复流程，适用于各种类型的退化，如低分辨率、模糊、噪声、压缩伪影等，通过利用 2D 修复方法恢复多视角图像，并使用生成对抗网络（GANs）进行 NeRF 生成来更好地适应几何和外观的不一致性，展示了在量化和定性评估中优于其他特定单一任务的 3D 修复方法的卓越性能。