Reconstructing deformable tissues from endoscopic stereo videos is essential
in many downstream surgical applications. However, existing methods suffer from
slow inference speed, which greatly limits their practical use. In this paper,
we introduce EndoGaussian, a real-time surgical scene reconstruction framework
that builds on 3D Gaussian Splatting. Our framework represents dynamic surgical
scenes as canonical Gaussians and a time-dependent deformation field, which
predicts Gaussian deformations at novel timestamps. Due to the efficient
Gaussian representation and parallel rendering pipeline, our framework
significantly accelerates the rendering speed compared to previous methods. In
addition, we design the deformation field as the combination of a lightweight
encoding voxel and an extremely tiny MLP, allowing for efficient Gaussian
tracking with a minor rendering burden. Furthermore, we design a holistic
Gaussian initialization method to fully leverage the surface distribution
prior, achieved by searching informative points from across the input image
sequence. Experiments on public endoscope datasets demonstrate that our method
can achieve real-time rendering speed (195 FPS real-time, 100$\times$ gain)
while maintaining the state-of-the-art reconstruction quality (35.925 PSNR) and
the fastest training speed (within 2 min/scene), showing significant promise
for intraoperative surgery applications. Code is available at:
https://yifliu3.github.io/EndoGaussian/.

通过使用 3D 高斯泼洒方法，我们引入了实时外科手术场景重建框架 EndoGaussian，能够在现实时间内实现手术场景的重建，实现了很大的渲染加速，同时在保持最新的重建质量和最快的训练速度方面表现出显著的优势。