Recent advances in neural rendering have enabled highly photorealistic 3D
scene reconstruction and novel view synthesis. Despite this progress, current
state-of-the-art methods struggle to reconstruct high frequency detail, due to
factors such as a low-frequency bias of radiance fields and inaccurate camera
calibration. One approach to mitigate this issue is to enhance images
post-rendering. 2D enhancers can be pre-trained to recover some detail but are
agnostic to scene geometry and do not easily generalize to new distributions of
image degradation. Conversely, existing 3D enhancers are able to transfer
detail from nearby training images in a generalizable manner, but suffer from
inaccurate camera calibration and can propagate errors from the geometry into
rendered images. We propose a neural rendering enhancer, RoGUENeRF, which
exploits the best of both paradigms. Our method is pre-trained to learn a
general enhancer while also leveraging information from nearby training images
via robust 3D alignment and geometry-aware fusion. Our approach restores
high-frequency textures while maintaining geometric consistency and is also
robust to inaccurate camera calibration. We show that RoGUENeRF substantially
enhances the rendering quality of a wide range of neural rendering baselines,
e.g. improving the PSNR of MipNeRF360 by 0.63dB and Nerfacto by 1.34dB on the
real world 360v2 dataset.

提出了 RoGUENeRF 神经渲染增强器，利用 2D 和 3D 技术来提高渲染质量和几何一致性，修复高频纹理并解决摄像机标定问题。