Neural Radiance Fields (NeRF) are able to reconstruct scenes with
unprecedented fidelity, and various recent works have extended NeRF to handle
dynamic scenes. A common approach to reconstruct such non-rigid scenes is
through the use of a learned deformation field mapping from coordinates in each
input image into a canonical template coordinate space. However, these
deformation-based approaches struggle to model changes in topology, as
topological changes require a discontinuity in the deformation field, but these
deformation fields are necessarily continuous. We address this limitation by
lifting NeRFs into a higher dimensional space, and by representing the 5D
radiance field corresponding to each individual input image as a slice through
this "hyper-space". Our method is inspired by level set methods, which model
the evolution of surfaces as slices through a higher dimensional surface. We
evaluate our method on two tasks: (i) interpolating smoothly between "moments",
i.e., configurations of the scene, seen in the input images while maintaining
visual plausibility, and (ii) novel-view synthesis at fixed moments. We show
that our method, which we dub HyperNeRF, outperforms existing methods on both
tasks. Compared to Nerfies, HyperNeRF reduces average error rates by 4.1% for
interpolation and 8.6% for novel-view synthesis, as measured by LPIPS.
Additional videos, results, and visualizations are available at
this https URL

本文提出了一种新的三维场景重建方法 HyperNeRF，通过将 NeRF 升维，将每个输入图像对应的五维辐射场表示为该 “超空间” 中的一片，从而解决了现有变形场模型在处理拓扑变化时的困难。实验表明，HyperNeRF 在平滑插值和新视角合成方面皆优于 Nerfies，平均误差率分别降低了 4.1％和 8.6％。