Neural networks can represent and accurately reconstruct radiance fields for
static 3D scenes (e.g., NeRF). Several works extend these to dynamic scenes
captured with monocular video, with promising performance. However, the
monocular setting is known to be an under-constrained problem, and so methods
rely on data-driven priors for reconstructing dynamic content. We replace these
priors with measurements from a time-of-flight (ToF) camera, and introduce a
neural representation based on an image formation model for continuous-wave ToF
cameras. Instead of working with processed depth maps, we model the raw ToF
sensor measurements to improve reconstruction quality and avoid issues with low
reflectance regions, multi-path interference, and a sensor's limited
unambiguous depth range. We show that this approach improves robustness of
dynamic scene reconstruction to erroneous calibration and large motions, and
discuss the benefits and limitations of integrating RGB+ToF sensors that are
now available on modern smartphones.

本研究提出了一种基于连续波 ToF 相机的神经表征模型，改进了动态场景的重建质量及鲁棒性，讨论了将 RGB+ToF 传感器集成到现代智能手机上的益处和局限性。

TöRF：用于动态场景视图合成的飞行时间辐射场

TöRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesis

Range images captured by Time-of-Flight (ToF) cameras are corrupted with
multipath distortions due to interaction between modulated light signals and
scenes. The interaction is often complicated, which makes a model-based
solution elusive. We propose a learning-based approach for removing the
multipath distortions for a ToF camera in a robotic arm setup. Our approach is
based on deep learning. We use the robotic arm to automatically collect a large
amount of ToF range images containing various multipath distortions. The
training images are automatically labeled by leveraging a high precision
structured light sensor available only in the training time. In the test time,
we apply the learned model to remove the multipath distortions. This allows our
robotic arm setup to enjoy the speed and compact form of the ToF camera without
compromising with its range measurement errors. We conduct extensive
experimental validations and compare the proposed method to several baseline
algorithms. The experiment results show that our method achieves 55% error
reduction in range estimation and largely outperforms the baseline algorithms.

本研究提出了一种基于深度学习的学习方法，采用机械臂自动地采集大量的 ToF 范围图像，并在训练时间利用高精度结构光传感器自动标记训练图像，从而去除 ToF 相机中的多路径失真。在实验验证中，我们的方法实现了 55％的测距误差减小，并且比其他基线算法表现更好。