Empowering 3D Gaussian Splatting with generalization ability is appealing.
However, existing generalizable 3D Gaussian Splatting methods are largely
confined to narrow-range interpolation between stereo images due to their heavy
backbones, thus lacking the ability to accurately localize 3D Gaussian and
support free-view synthesis across wide view range. In this paper, we present a
novel framework FreeSplat that is capable of reconstructing geometrically
consistent 3D scenes from long sequence input towards free-view
synthesis.Specifically, we firstly introduce Low-cost Cross-View Aggregation
achieved by constructing adaptive cost volumes among nearby views and
aggregating features using a multi-scale structure. Subsequently, we present
the Pixel-wise Triplet Fusion to eliminate redundancy of 3D Gaussians in
overlapping view regions and to aggregate features observed across multiple
views. Additionally, we propose a simple but effective free-view training
strategy that ensures robust view synthesis across broader view range
regardless of the number of views. Our empirical results demonstrate
state-of-the-art novel view synthesis peformances in both novel view rendered
color maps quality and depth maps accuracy across different numbers of input
views. We also show that FreeSplat performs inference more efficiently and can
effectively reduce redundant Gaussians, offering the possibility of
feed-forward large scene reconstruction without depth priors.

通过引入低成本跨视聚集、像素级三元组融合和简单有效的自由视图训练策略，我们提出了一个能够从长序列输入中重建几何一致的 3D 场景并实现自由视图合成的新方法 FreeSplat，研究结果表明其在不同数量的输入视图下，在新视图生成的颜色贴图质量和深度图准确性方面都达到了最新水平，并且 FreeSplat 的推断效率更高，能够有效减少冗余的高斯函数，为无需深度先验的大场景重建提供了可能。

FreeSplat: 室内场景的自由视角综合的通用化三维高斯点采样

FreeSplat: Generalizable 3D Gaussian Splatting Towards Free-View  Synthesis of Indoor Scenes

In this work, we propose a method to address the challenge of rendering a 3D
human from a single image in a free-view manner. Some existing approaches could
achieve this by using generalizable pixel-aligned implicit fields to
reconstruct a textured mesh of a human or by employing a 2D diffusion model as
guidance with the Score Distillation Sampling (SDS) method, to lift the 2D
image into 3D space. However, a generalizable implicit field often results in
an over-smooth texture field, while the SDS method tends to lead to a
texture-inconsistent novel view with the input image. In this paper, we
introduce a texture-consistent back view synthesis module that could transfer
the reference image content to the back view through depth and text-guided
attention injection. Moreover, to alleviate the color distortion that occurs in
the side region, we propose a visibility-aware patch consistency regularization
for texture mapping and refinement combined with the synthesized back view
texture. With the above techniques, we could achieve high-fidelity and
texture-consistent human rendering from a single image. Experiments conducted
on both real and synthetic data demonstrate the effectiveness of our method and
show that our approach outperforms previous baseline methods.

本文提出了一种从单幅图像中渲染出具有一致纹理的 3D 人物的方法，并通过深度和文本引导的关注注入将参考图像内容转移到背景视图，同时结合合成的背景视图纹理实现可见性感知的贴图和细化，通过实验证明了方法的有效性并展示了超越基准方法的成果。