3D Gaussian Splatting has emerged as a very promising scene representation,
achieving state-of-the-art quality in novel view synthesis significantly faster
than competing alternatives. However, its use of spherical harmonics to
represent scene colors limits the expressivity of 3D Gaussians and, as a
consequence, the capability of the representation to generalize as we move away
from the training views. In this paper, we propose to encode the color
information of 3D Gaussians into per-Gaussian feature vectors, which we denote
as Feature Splatting (FeatSplat). To synthesize a novel view, Gaussians are
first "splatted" into the image plane, then the corresponding feature vectors
are alpha-blended, and finally the blended vector is decoded by a small MLP to
render the RGB pixel values. To further inform the model, we concatenate a
camera embedding to the blended feature vector, to condition the decoding also
on the viewpoint information. Our experiments show that these novel model for
encoding the radiance considerably improves novel view synthesis for low
overlap views that are distant from the training views. Finally, we also show
the capacity and convenience of our feature vector representation,
demonstrating its capability not only to generate RGB values for novel views,
but also their per-pixel semantic labels. We will release the code upon
acceptance.
Keywords: Gaussian Splatting, Novel View Synthesis, Feature Splatting

利用特征喷溅（FeatSplat）将三维高斯场的颜色信息编码到每个高斯特征向量中，进而通过小型 MLP 解码融合的特征向量以生成 RGB 像素值，且通过叠加相机嵌入来条件解码以包含视角信息。实验证明，该方法显著提高了对训练视图之外远离的低重叠视图的新视图合成，并展示了特征向量表示的能力不仅可以生成新视图的 RGB 值，还可以生成每个像素的语义标签。

特征分割以减少重叠度的优化新视角合成

Feature Splatting for Better Novel View Synthesis with Low Overlap

Scene representations using 3D Gaussian primitives have produced excellent
results in modeling the appearance of static and dynamic 3D scenes. Many
graphics applications, however, demand the ability to manipulate both the
appearance and the physical properties of objects. We introduce Feature
Splatting, an approach that unifies physics-based dynamic scene synthesis with
rich semantics from vision language foundation models that are grounded by
natural language. Our first contribution is a way to distill high-quality,
object-centric vision-language features into 3D Gaussians, that enables
semi-automatic scene decomposition using text queries. Our second contribution
is a way to synthesize physics-based dynamics from an otherwise static scene
using a particle-based simulator, in which material properties are assigned
automatically via text queries. We ablate key techniques used in this pipeline,
to illustrate the challenge and opportunities in using feature-carrying 3D
Gaussians as a unified format for appearance, geometry, material properties and
semantics grounded on natural language. Project website:
this https URL

使用 3D 高斯原始物体来建模静态和动态 3D 场景的外观已经取得了优秀的结果。本研究提出了一种统一物理动态场景合成和由自然语言支持的丰富语义的特征平面化方法。通过将高质量的面向对象视觉语言特征整合为 3D 高斯模型，能够利用文本查询实现半自动场景分解。同时，通过基于粒子的模拟器从静态场景中合成物理动力学，通过文本查询自动分配材料属性。本研究在这一流程中消去了关键技术，以说明将携带特征的 3D 高斯模型作为外观、几何、材料属性和基于自然语言的语义的统一格式的挑战和机遇。