Photorealistic reconstruction relying on 3D Gaussian Splatting has shown
promising potential in robotics. However, the current 3D Gaussian Splatting
system only supports radiance field reconstruction using undistorted
perspective images. In this paper, we present OmniGS, a novel omnidirectional
Gaussian splatting system, to take advantage of omnidirectional images for fast
radiance field reconstruction. Specifically, we conduct a theoretical analysis
of spherical camera model derivatives in 3D Gaussian Splatting. According to
the derivatives, we then implement a new GPU-accelerated omnidirectional
rasterizer that directly splats 3D Gaussians onto the equirectangular screen
space for omnidirectional image rendering. As a result, we realize
differentiable optimization of the radiance field without the requirement of
cube-map rectification or tangent-plane approximation. Extensive experiments
conducted in egocentric and roaming scenarios demonstrate that our method
achieves state-of-the-art reconstruction quality and high rendering speed using
omnidirectional images. To benefit the research community, the code will be
made publicly available once the paper is published.

本文介绍了一种新的全向高斯散乱系统 ——OmniGS，它利用全向图像进行快速辐射场重建，通过对全向相机模型导数的理论分析，在 GPU 加速的全向光栅化器中直接对高斯函数进行散乱，实现了辐射场的可微优化，无需立方体贴图矫正或切平面近似。实验结果表明，该方法在自我中心和移动场景中实现了最先进的重建质量和高渲染速度，并在论文发表后将公开提供代码。

OmniGS：使用全向图像进行快速辐射场重建的全向高斯喷洒

OmniGS: Omnidirectional Gaussian Splatting for Fast Radiance Field  Reconstruction using Omnidirectional Images

We present the first method capable of photorealistically reconstructing
deformable scenes using photos/videos captured casually from mobile phones. Our
approach augments neural radiance fields (NeRF) by optimizing an additional
continuous volumetric deformation field that warps each observed point into a
canonical 5D NeRF. We observe that these NeRF-like deformation fields are prone
to local minima, and propose a coarse-to-fine optimization method for
coordinate-based models that allows for more robust optimization. By adapting
principles from geometry processing and physical simulation to NeRF-like
models, we propose an elastic regularization of the deformation field that
further improves robustness. We show that our method can turn casually captured
selfie photos/videos into deformable NeRF models that allow for photorealistic
renderings of the subject from arbitrary viewpoints, which we dub "nerfies." We
evaluate our method by collecting time-synchronized data using a rig with two
mobile phones, yielding train/validation images of the same pose at different
viewpoints. We show that our method faithfully reconstructs non-rigidly
deforming scenes and reproduces unseen views with high fidelity.

描述了一种基于神经辐射场的方法，通过优化连续的体形变场使移动设备拍摄的照片 / 视频重建变形场景，旨在生成高度逼真的渲染图像。实验表明，该方法可使用手机拍摄的自拍照片 / 视频转换成可变形的 NeRF 模型，并在任意视点产生高保真度的视图。