Creating 4D fields of Gaussian Splatting from images or videos is a
challenging task due to its under-constrained nature. While the optimization
can draw photometric reference from the input videos or be regulated by
generative models, directly supervising Gaussian motions remains underexplored.
In this paper, we introduce a novel concept, Gaussian flow, which connects the
dynamics of 3D Gaussians and pixel velocities between consecutive frames. The
Gaussian flow can be efficiently obtained by splatting Gaussian dynamics into
the image space. This differentiable process enables direct dynamic supervision
from optical flow. Our method significantly benefits 4D dynamic content
generation and 4D novel view synthesis with Gaussian Splatting, especially for
contents with rich motions that are hard to be handled by existing methods. The
common color drifting issue that happens in 4D generation is also resolved with
improved Guassian dynamics. Superior visual quality on extensive experiments
demonstrates our method's effectiveness. Quantitative and qualitative
evaluations show that our method achieves state-of-the-art results on both
tasks of 4D generation and 4D novel view synthesis. Project page:
this https URL

通过将高斯动态投射到图像空间中，我们引入了一个新的概念，高斯流，它连接了 3D 高斯和相邻帧之间的像素速度，从而实现了光流的直接动态监督，显著提高了 4D 动态内容生成和 4D 新视图合成的视觉质量，并解决了 4D 生成过程中常见的颜色移动问题。

高斯流：基于高斯动力学的四维内容创建

GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation

In estimation theory, the Kushner equation provides the evolution of the
probability density of the state of a dynamical system given continuous-time
observations. Building upon our recent work, we propose a new way to
approximate the solution of the Kushner equation through tractable variational
Gaussian approximations of two proximal losses associated with the propagation
and Bayesian update of the probability density. The first is a proximal loss
based on the Wasserstein metric and the second is a proximal loss based on the
Fisher metric. The solution to this last proximal loss is given by implicit
updates on the mean and covariance that we proposed earlier. These two
variational updates can be fused and shown to satisfy a set of stochastic
differential equations on the Gaussian's mean and covariance matrix. This
Gaussian flow is consistent with the Kalman-Bucy and Riccati flows in the
linear case and generalize them in the nonlinear one.

基于可变分高斯逼近，本文提出了一种近似计算 Kushner 方程解的新方法，该方法通过与概率密度的传播和贝叶斯更新相关的两个接近损失函数的可计算高斯逼近来实现。这两个变分更新可以被融合，并满足高斯的均值和协方差矩阵上的一组随机微分方程，该高斯流与线性情况下的 Kalman-Bucy 和 Riccati 流一致，并在非线性情况下进行了推广。