We propose GS-LRM, a scalable large reconstruction model that can predict
high-quality 3D Gaussian primitives from 2-4 posed sparse images in 0.23
seconds on single A100 GPU. Our model features a very simple transformer-based
architecture; we patchify input posed images, pass the concatenated multi-view
image tokens through a sequence of transformer blocks, and decode final
per-pixel Gaussian parameters directly from these tokens for differentiable
rendering. In contrast to previous LRMs that can only reconstruct objects, by
predicting per-pixel Gaussians, GS-LRM naturally handles scenes with large
variations in scale and complexity. We show that our model can work on both
object and scene captures by training it on Objaverse and RealEstate10K
respectively. In both scenarios, the models outperform state-of-the-art
baselines by a wide margin. We also demonstrate applications of our model in
downstream 3D generation tasks. Our project webpage is available at:
this https URL .

我们提出了 GS-LRM，一个可扩展的大型重建模型，可以在单个 A100 GPU 上从 2-4 个稀疏图像中预测高质量的 3D 高斯原语，并在 0.23 秒内完成。我们的模型采用了非常简单的基于 transformer 的架构；我们对输入的图像进行了分块处理，通过一系列的 transformer 块将连接的多视图图像令牌传递，并直接从这些令牌解码出每像素的高斯参数以进行可微渲染。与之前仅能重建对象的低秩模型不同，GS-LRM 通过预测每像素的高斯分布，能够自然地处理具有不同尺度和复杂性的场景。我们展示了我们的模型可以适用于对象和场景捕捉，并通过在 Objaverse 和 RealEstate10K 上进行训练，在这两种情景下，我们的模型都比现有的最先进方法表现得更好。我们还展示了我们模型在下游 3D 生成任务中的应用。我们的项目网页位于：this https URL。

GS-LRM: 三维高斯喷射的大规模重建模型

GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting

Scene representations using 3D Gaussian primitives have produced excellent
results in modeling the appearance of static and dynamic 3D scenes. Many
graphics applications, however, demand the ability to manipulate both the
appearance and the physical properties of objects. We introduce Feature
Splatting, an approach that unifies physics-based dynamic scene synthesis with
rich semantics from vision language foundation models that are grounded by
natural language. Our first contribution is a way to distill high-quality,
object-centric vision-language features into 3D Gaussians, that enables
semi-automatic scene decomposition using text queries. Our second contribution
is a way to synthesize physics-based dynamics from an otherwise static scene
using a particle-based simulator, in which material properties are assigned
automatically via text queries. We ablate key techniques used in this pipeline,
to illustrate the challenge and opportunities in using feature-carrying 3D
Gaussians as a unified format for appearance, geometry, material properties and
semantics grounded on natural language. Project website:
this https URL

使用 3D 高斯原始物体来建模静态和动态 3D 场景的外观已经取得了优秀的结果。本研究提出了一种统一物理动态场景合成和由自然语言支持的丰富语义的特征平面化方法。通过将高质量的面向对象视觉语言特征整合为 3D 高斯模型，能够利用文本查询实现半自动场景分解。同时，通过基于粒子的模拟器从静态场景中合成物理动力学，通过文本查询自动分配材料属性。本研究在这一流程中消去了关键技术，以说明将携带特征的 3D 高斯模型作为外观、几何、材料属性和基于自然语言的语义的统一格式的挑战和机遇。