Omnidirectional cameras are extensively used in various applications to
provide a wide field of vision. However, they face a challenge in synthesizing
novel views due to the inevitable presence of dynamic objects, including the
photographer, in their wide field of view. In this paper, we introduce a new
approach called Omnidirectional Local Radiance Fields (OmniLocalRF) that can
render static-only scene views, removing and inpainting dynamic objects
simultaneously. Our approach combines the principles of local radiance fields
with the bidirectional optimization of omnidirectional rays. Our input is an
omnidirectional video, and we evaluate the mutual observations of the entire
angle between the previous and current frames. To reduce ghosting artifacts of
dynamic objects and inpaint occlusions, we devise a multi-resolution motion
mask prediction module. Unlike existing methods that primarily separate dynamic
components through the temporal domain, our method uses multi-resolution neural
feature planes for precise segmentation, which is more suitable for long
360-degree videos. Our experiments validate that OmniLocalRF outperforms
existing methods in both qualitative and quantitative metrics, especially in
scenarios with complex real-world scenes. In particular, our approach
eliminates the need for manual interaction, such as drawing motion masks by
hand and additional pose estimation, making it a highly effective and efficient
solution.

本文介绍了一种称为 OmniLocalRF 的新方法，能够同时渲染仅静态场景视图、删除和修补动态对象。本方法使用局部辐射场原理和全向射线的双向优化原则，以全向视频作为输入，评估前一帧和当前帧之间的相互观察。为了减少动态对象的幽灵状伪影和修复遮挡，我们设计了一个多分辨率运动蒙版预测模块，通过多分辨率神经特征平面进行精确分割，特别适用于长 360 度视频。我们的实验证实了 OmniLocalRF 在定性和定量评估指标上优于现有方法，尤其是在复杂的真实场景中。特别是，我们的方法消除了手动交互的需求，例如手动绘制运动蒙版和其他姿态估计，因此是一个高效且高效的解决方案。