We tackle the challenge of efficiently reconstructing a 3D asset from a
single image with growing demands for automated 3D content creation pipelines.
Previous methods primarily rely on Score Distillation Sampling (SDS) and Neural
Radiance Fields (NeRF). Despite their significant success, these approaches
encounter practical limitations due to lengthy optimization and considerable
memory usage. In this report, we introduce Gamba, an end-to-end amortized 3D
reconstruction model from single-view images, emphasizing two main insights:
(1) 3D representation: leveraging a large number of 3D Gaussians for an
efficient 3D Gaussian splatting process; (2) Backbone design: introducing a
Mamba-based sequential network that facilitates context-dependent reasoning and
linear scalability with the sequence (token) length, accommodating a
substantial number of Gaussians. Gamba incorporates significant advancements in
data preprocessing, regularization design, and training methodologies. We
assessed Gamba against existing optimization-based and feed-forward 3D
generation approaches using the real-world scanned OmniObject3D dataset. Here,
Gamba demonstrates competitive generation capabilities, both qualitatively and
quantitatively, while achieving remarkable speed, approximately 0.6 second on a
single NVIDIA A100 GPU.

我们通过使用大量的三维高斯模型实现了高效的三维资产重建，引入了一种基于序列网络的设计来进行上下文相关的推理和线性可伸缩处理，从而实现了优秀的生成能力和惊人的速度。

Gamba：将高斯点渲染与 Mamba 相结合用于单视角三维重建

Gamba: Marry Gaussian Splatting with Mamba for single view 3D  reconstruction

We introduce GRM, a large-scale reconstructor capable of recovering a 3D
asset from sparse-view images in around 0.1s. GRM is a feed-forward
transformer-based model that efficiently incorporates multi-view information to
translate the input pixels into pixel-aligned Gaussians, which are unprojected
to create a set of densely distributed 3D Gaussians representing a scene.
Together, our transformer architecture and the use of 3D Gaussians unlock a
scalable and efficient reconstruction framework. Extensive experimental results
demonstrate the superiority of our method over alternatives regarding both
reconstruction quality and efficiency. We also showcase the potential of GRM in
generative tasks, i.e., text-to-3D and image-to-3D, by integrating it with
existing multi-view diffusion models. Our project website is at:
this https URL

我们介绍了 GRM，一个能够从稀疏视图图像中在大约 0.1 秒内恢复 3D 资产的大规模重构器。GRM 是一个前馈变换器模型，能够高效地将多视图信息结合起来，将输入像素翻译成像素对齐的高斯分布，通过反投影生成一系列密集分布的 3D 高斯分布来表示场景。我们的变换器架构和使用 3D 高斯分布的方法共同构成了一个可扩展且高效的重构框架。广泛的实验结果证明，我们的方法在重构质量和效率方面优于其他替代方法。我们还展示了 GRM 在生成任务中的潜力，即将其与现有的多视图扩散模型集成，如文本到 3D 和图像到 3D。有关我们的项目网站，请访问：this https URL