Differentiable rendering techniques have recently shown promising results for free-viewpoint video synthesis of characters. However, such methods, either Gaussian Splatting or neural implicit rendering, typically necessitate per-subject optimization which does not meet the requirement of real-time rendering in an interactive application. We propose a generalizable Gaussian Splatting approach for high-resolution image rendering under a sparse-view camera setting. To this end, we introduce Gaussian parameter maps defined on the source views and directly regress Gaussian properties for instant novel view synthesis without any fine-tuning or optimization. We train our Gaussian parameter regression module on human-only data or human-scene data, jointly with a depth estimation module to lift 2D parameter maps to 3D space. The proposed framework is fully differentiable with both depth and rendering supervision or with only rendering supervision. We further introduce a regularization term and an epipolar attention mechanism to preserve geometry consistency between two source views, especially when neglecting depth supervision. Experiments on several datasets demonstrate that our method outperforms state-of-the-art methods while achieving an exceeding rendering speed.

本研究解决了传统高分辨率图像渲染在实时交互应用中的局限性，尤其是在稀疏视角条件下。提出了一种新颖的高斯点云方法，直接回归高斯属性以实现即时新视图合成，显著提高了渲染速度并保持几何一致性。实验结果表明，该方法在多个数据集上优于现有技术，并具有潜在的广泛应用价值。

GPS-Gaussian+: 泛化像素级3D高斯点云快速渲染人场景的技术