We introduce a method to render Neural Radiance Fields (NeRFs) in real time
using PlenOctrees, an octree-based 3D representation which supports
view-dependent effects. Our method can render 800x800 images at more than 150
FPS, which is over 3000 times faster than conventional NeRFs. We do so without
sacrificing quality while preserving the ability of NeRFs to perform
free-viewpoint rendering of scenes with arbitrary geometry and view-dependent
effects. Real-time performance is achieved by pre-tabulating the NeRF into a
PlenOctree. In order to preserve view-dependent effects such as specularities,
we factorize the appearance via closed-form spherical basis functions.
Specifically, we show that it is possible to train NeRFs to predict a spherical
harmonic representation of radiance, removing the viewing direction as an input
to the neural network. Furthermore, we show that PlenOctrees can be directly
optimized to further minimize the reconstruction loss, which leads to equal or
better quality compared to competing methods. Moreover, this octree
optimization step can be used to reduce the training time, as we no longer need
to wait for the NeRF training to converge fully. Our real-time neural rendering
approach may potentially enable new applications such as 6-DOF industrial and
product visualizations, as well as next generation AR/VR systems. PlenOctrees
are amenable to in-browser rendering as well; please visit the project page for
the interactive online demo, as well as video and code:
this https URL

该研究提出了一种使用 PlenOctrees 进行神经辐射场（NeRFs）实时渲染的方法，以支持视角相关的效果，该方法可在保证渲染质量的情况下渲染 800x800 像素以上的图像，并能在 150 FPS 以上的速度下运行。