Neural fields, a category of neural networks trained to represent
high-frequency signals, have gained significant attention in recent years due
to their impressive performance in modeling complex 3D data, especially large
neural signed distance (SDFs) or radiance fields (NeRFs) via a single
multi-layer perceptron (MLP). However, despite the power and simplicity of
representing signals with an MLP, these methods still face challenges when
modeling large and complex temporal signals due to the limited capacity of
MLPs. In this paper, we propose an effective approach to address this
limitation by incorporating temporal residual layers into neural fields, dubbed
ResFields, a novel class of networks specifically designed to effectively
represent complex temporal signals. We conduct a comprehensive analysis of the
properties of ResFields and propose a matrix factorization technique to reduce
the number of trainable parameters and enhance generalization capabilities.
Importantly, our formulation seamlessly integrates with existing techniques and
consistently improves results across various challenging tasks: 2D video
approximation, dynamic shape modeling via temporal SDFs, and dynamic NeRF
reconstruction. Lastly, we demonstrate the practical utility of ResFields by
showcasing its effectiveness in capturing dynamic 3D scenes from sparse sensory
inputs of a lightweight capture system.

提出了一种将时间残差层引入神经场的有效方法，用于表示复杂的时间信号，通过降低可训练参数数量和增强泛化能力的矩阵分解技术，实现了对 2D 视频逼近、通过时间 SDFs 进行动态形状建模和动态 NeRF 重建等各种具有挑战性任务中结果的稳定提升，并通过展示其在轻量级捕获系统的稀疏传感输入中捕获动态 3D 场景的实际效果证明了 ResFields 的实用性。