Modeling Neural Radiance Fields for fast-moving deformable objects from
visual data alone is a challenging problem. A major issue arises due to the
high deformation and low acquisition rates. To address this problem, we propose
to use event cameras that offer very fast acquisition of visual change in an
asynchronous manner. In this work, we develop a novel method to model the
deformable neural radiance fields using RGB and event cameras. The proposed
method uses the asynchronous stream of events and calibrated sparse RGB frames.
In our setup, the camera pose at the individual events required to integrate
them into the radiance fields remains unknown. Our method jointly optimizes
these poses and the radiance field. This happens efficiently by leveraging the
collection of events at once and actively sampling the events during learning.
Experiments conducted on both realistically rendered graphics and real-world
datasets demonstrate a significant benefit of the proposed method over the
state-of-the-art and the compared baseline.
This shows a promising direction for modeling deformable neural radiance
fields in real-world dynamic scenes.

使用 RGB 和事件相机以及异步的事件流与校正的稀疏 RGB 帧的组合，共同优化事件相机的位置和辐射场，以建模可变形的神经辐射场。该方法在处理高变形和低采样率的情况下显示出显著优势，并在真实世界数据集上进行了实验证明其对建模动态场景中的可变形神经辐射场具有良好的应用前景。