We present an end-to-end system for the high-fidelity capture, model
reconstruction, and real-time rendering of walkable spaces in virtual reality
using neural radiance fields. To this end, we designed and built a custom
multi-camera rig to densely capture walkable spaces in high fidelity and with
multi-view high dynamic range images in unprecedented quality and density. We
extend instant neural graphics primitives with a novel perceptual color space
for learning accurate HDR appearance, and an efficient mip-mapping mechanism
for level-of-detail rendering with anti-aliasing, while carefully optimizing
the trade-off between quality and speed. Our multi-GPU renderer enables
high-fidelity volume rendering of our neural radiance field model at the full
VR resolution of dual 2K$\times$2K at 36 Hz on our custom demo machine. We
demonstrate the quality of our results on our challenging high-fidelity
datasets, and compare our method and datasets to existing baselines. We release
our dataset on our project website.

我们提出了一个端到端系统，用于在虚拟现实中高保真地捕捉、重建模型并进行实时渲染可行走空间，使用神经辐射场。

VR-NeRF：高保真虚拟行走空间

VR-NeRF: High-Fidelity Virtualized Walkable Spaces

Advances in neural fields are enabling high-fidelity capture of the shape and
appearance of static and dynamic scenes. However, their capabilities lag behind
those offered by representations such as pixels or meshes due to algorithmic
challenges and the lack of large-scale real-world datasets. We address the
dataset limitation with DiVA-360, a real-world 360 dynamic visual-audio dataset
with synchronized multimodal visual, audio, and textual information about
table-scale scenes. It contains 46 dynamic scenes, 30 static scenes, and 95
static objects spanning 11 categories captured using a new hardware system
using 53 RGB cameras at 120 FPS and 6 microphones for a total of 8.6M image
frames and 1360 s of dynamic data. We provide detailed text descriptions for
all scenes, foreground-background segmentation masks, category-specific 3D pose
alignment for static objects, as well as metrics for comparison. Our data,
hardware and software, and code are available at this https URL

通过 DiVA-360 数据集，我们解决了高保真捕捉静态和动态场景形状和外观的算法挑战和缺乏大规模现实世界数据集的问题。该数据集包含 46 个动态场景、30 个静态场景和 95 个静态物体的图像帧和动态数据，使用 53 台 RGB 相机和 6 个麦克风采集。我们提供了详细的文本描述、前景 - 背景分割掩码、静态物体的类别特定的 3D 姿势对齐，以及用于比较的度量。