Constructing photo-realistic Free-Viewpoint Videos (FVVs) of dynamic scenes
from multi-view videos remains a challenging endeavor. Despite the remarkable
advancements achieved by current neural rendering techniques, these methods
generally require complete video sequences for offline training and are not
capable of real-time rendering. To address these constraints, we introduce
3DGStream, a method designed for efficient FVV streaming of real-world dynamic
scenes. Our method achieves fast on-the-fly per-frame reconstruction within 12
seconds and real-time rendering at 200 FPS. Specifically, we utilize 3D
Gaussians (3DGs) to represent the scene. Instead of the na\"ive approach of
directly optimizing 3DGs per-frame, we employ a compact Neural Transformation
Cache (NTC) to model the translations and rotations of 3DGs, markedly reducing
the training time and storage required for each FVV frame. Furthermore, we
propose an adaptive 3DG addition strategy to handle emerging objects in dynamic
scenes. Experiments demonstrate that 3DGStream achieves competitive performance
in terms of rendering speed, image quality, training time, and model storage
when compared with state-of-the-art methods.

3DGStream 是一种用于实时流式传输现实世界动态场景的方法，通过使用 3D 高斯（3DGs）表示场景，并利用紧凑的神经转换缓存（NTC）来建模 3DGs 的平移和旋转，从而实现了每帧 12 秒内的快速即时重建和 200 FPS 的实时渲染，同时提出了自适应 3DG 添加策略来处理动态场景中出现的新物体。实验证明，与最先进的方法相比，3DGStream 在渲染速度、图像质量、训练时间和模型存储方面表现出了竞争力。