Robot-assisted minimally invasive surgery benefits from enhancing dynamic
scene reconstruction, as it improves surgical outcomes. While Neural Radiance
Fields (NeRF) have been effective in scene reconstruction, their slow inference
speeds and lengthy training durations limit their applicability. To overcome
these limitations, 3D Gaussian Splatting (3D-GS) based methods have emerged as
a recent trend, offering rapid inference capabilities and superior 3D quality.
However, these methods still struggle with under-reconstruction in both static
and dynamic scenes. In this paper, we propose HFGS, a novel approach for
deformable endoscopic reconstruction that addresses these challenges from
spatial and temporal frequency perspectives. Our approach incorporates
deformation fields to better handle dynamic scenes and introduces Spatial
High-Frequency Emphasis Reconstruction (SHF) to minimize discrepancies in
spatial frequency spectra between the rendered image and its ground truth.
Additionally, we introduce Temporal High-Frequency Emphasis Reconstruction
(THF) to enhance dynamic awareness in neural rendering by leveraging flow
priors, focusing optimization on motion-intensive parts. Extensive experiments
on two widely used benchmarks demonstrate that HFGS achieves superior rendering
quality. Our code will be available.

提出了一种名为 HFGS 的新方法，通过空间和时间频率角度处理动态场景的可变形内窥镜重建问题，并引入了空间高频重建和时间高频重建技术来提高神经渲染的动态感知和 3D 质量。

HFGS: 四维高斯完全投影，着重于内窥镜场景重建中的空间和时间高频分量

HFGS: 4D Gaussian Splatting with Emphasis on Spatial and Temporal  High-Frequency Components for Endoscopic Scene Reconstruction

Recent developments in neural rendering techniques have greatly enhanced the
rendering of photo-realistic 3D scenes across both academic and commercial
fields. The latest method, known as 3D Gaussian Splatting (3D-GS), has set new
benchmarks for rendering quality and speed. Nevertheless, the limitations of
3D-GS become pronounced in synthesizing new viewpoints, especially for views
that greatly deviate from those seen during training. Additionally, issues such
as dilation and aliasing arise when zooming in or out. These challenges can all
be traced back to a single underlying issue: insufficient sampling. In our
paper, we present a bootstrapping method that significantly addresses this
problem. This approach employs a diffusion model to enhance the rendering of
novel views using trained 3D-GS, thereby streamlining the training process. Our
results indicate that bootstrapping effectively reduces artifacts, as well as
clear enhancements on the evaluation metrics. Furthermore, we show that our
method is versatile and can be easily integrated, allowing various 3D
reconstruction projects to benefit from our approach.

我们的研究论文提出了一种基于扩散模型的引导方法，通过改进经过训练的 3D-GS 来增强新视角的渲染效果，显著减少了伪影，并在评估指标上获得明显的改进。此外，我们的方法具有通用性，可以轻松整合到各种 3D 重建项目中。

从 3D 高斯飞溅重构的 Bootstrap 3D 场景

Bootstrap 3D Reconstructed Scenes from 3D Gaussian Splatting

This paper presents GGRt, a novel approach to generalizable novel view
synthesis that alleviates the need for real camera poses, complexity in
processing high-resolution images, and lengthy optimization processes, thus
facilitating stronger applicability of 3D Gaussian Splatting (3D-GS) in
real-world scenarios. Specifically, we design a novel joint learning framework
that consists of an Iterative Pose Optimization Network (IPO-Net) and a
Generalizable 3D-Gaussians (G-3DG) model. With the joint learning mechanism,
the proposed framework can inherently estimate robust relative pose information
from the image observations and thus primarily alleviate the requirement of
real camera poses. Moreover, we implement a deferred back-propagation mechanism
that enables high-resolution training and inference, overcoming the resolution
constraints of previous methods. To enhance the speed and efficiency, we
further introduce a progressive Gaussian cache module that dynamically adjusts
during training and inference. As the first pose-free generalizable 3D-GS
framework, GGRt achieves inference at $\ge$ 5 FPS and real-time rendering at
$\ge$ 100 FPS. Through extensive experimentation, we demonstrate that our
method outperforms existing NeRF-based pose-free techniques in terms of
inference speed and effectiveness. It can also approach the real pose-based
3D-GS methods. Our contributions provide a significant leap forward for the
integration of computer vision and computer graphics into practical
applications, offering state-of-the-art results on LLFF, KITTI, and Waymo Open
datasets and enabling real-time rendering for immersive experiences.

这篇论文介绍了 GGRt，一种新颖的通用新视角合成方法，它减轻了对真实相机位姿、处理高分辨率图像的复杂性和冗长的优化过程的需要，从而促进了 3D 高斯喷溅（3D-GS）在实际场景中的更强适用性。