Creating 4D fields of Gaussian Splatting from images or videos is a
challenging task due to its under-constrained nature. While the optimization
can draw photometric reference from the input videos or be regulated by
generative models, directly supervising Gaussian motions remains underexplored.
In this paper, we introduce a novel concept, Gaussian flow, which connects the
dynamics of 3D Gaussians and pixel velocities between consecutive frames. The
Gaussian flow can be efficiently obtained by splatting Gaussian dynamics into
the image space. This differentiable process enables direct dynamic supervision
from optical flow. Our method significantly benefits 4D dynamic content
generation and 4D novel view synthesis with Gaussian Splatting, especially for
contents with rich motions that are hard to be handled by existing methods. The
common color drifting issue that happens in 4D generation is also resolved with
improved Guassian dynamics. Superior visual quality on extensive experiments
demonstrates our method's effectiveness. Quantitative and qualitative
evaluations show that our method achieves state-of-the-art results on both
tasks of 4D generation and 4D novel view synthesis. Project page:
this https URL

通过将高斯动态投射到图像空间中，我们引入了一个新的概念，高斯流，它连接了 3D 高斯和相邻帧之间的像素速度，从而实现了光流的直接动态监督，显著提高了 4D 动态内容生成和 4D 新视图合成的视觉质量，并解决了 4D 生成过程中常见的颜色移动问题。