We introduce GRM, a large-scale reconstructor capable of recovering a 3D
asset from sparse-view images in around 0.1s. GRM is a feed-forward
transformer-based model that efficiently incorporates multi-view information to
translate the input pixels into pixel-aligned Gaussians, which are unprojected
to create a set of densely distributed 3D Gaussians representing a scene.
Together, our transformer architecture and the use of 3D Gaussians unlock a
scalable and efficient reconstruction framework. Extensive experimental results
demonstrate the superiority of our method over alternatives regarding both
reconstruction quality and efficiency. We also showcase the potential of GRM in
generative tasks, i.e., text-to-3D and image-to-3D, by integrating it with
existing multi-view diffusion models. Our project website is at:
this https URL

我们介绍了 GRM，一个能够从稀疏视图图像中在大约 0.1 秒内恢复 3D 资产的大规模重构器。GRM 是一个前馈变换器模型，能够高效地将多视图信息结合起来，将输入像素翻译成像素对齐的高斯分布，通过反投影生成一系列密集分布的 3D 高斯分布来表示场景。我们的变换器架构和使用 3D 高斯分布的方法共同构成了一个可扩展且高效的重构框架。广泛的实验结果证明，我们的方法在重构质量和效率方面优于其他替代方法。我们还展示了 GRM 在生成任务中的潜力，即将其与现有的多视图扩散模型集成，如文本到 3D 和图像到 3D。有关我们的项目网站，请访问：this https URL

GRM：高效的大型高斯重建模型用于 3D 重建和生成

GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction  and Generation

This paper presents a novel two-stage approach for reconstructing human faces
from sparse-view images, a task made challenging by the unique geometry and
complex skin reflectance of each individual. Our method focuses on decomposing
key facial attributes, including geometry, diffuse reflectance, and specular
reflectance, from ambient light. Initially, we create a general facial template
from a diverse collection of individual faces, capturing essential geometric
and reflectance characteristics. Guided by this template, we refine each
specific face model in the second stage, which further considers the
interaction between geometry and reflectance, as well as the subsurface
scattering effects on facial skin. Our method enables the reconstruction of
high-quality facial representations from as few as three images, offering
improved geometric accuracy and reflectance detail. Through comprehensive
evaluations and comparisons, our method demonstrates superiority over existing
techniques. Our method effectively disentangles geometry and reflectance
components, leading to enhanced quality in synthesizing new views and opening
up possibilities for applications such as relighting and reflectance editing.
We will make the code publicly available.

本文提出了一种创新的两阶段方法，用于从稀疏视图图像重建人脸，该任务由于每个个体的独特几何和复杂皮肤反射而具有挑战性。我们的方法专注于从环境光中分解关键的面部属性，包括几何、漫反射和镜面反射。开始时，我们从多样化的个体脸部图像集创建一个通用的面部模板，捕捉关键的几何和反射特征。在第二阶段，我们在该模板的指导下对每个具体的人脸模型进行细化，进一步考虑几何和反射之间的相互作用，以及对面部皮肤的次表面散射效应。我们的方法能够从仅有的三幅图像中重建出高质量的面部表示，提供了改进的几何准确性和反射细节。通过全面的评估和比较，我们的方法表现出优越性，有效解离几何和反射成分，提高了合成新视图的质量，为再照明和反射编辑等应用开辟了可能。我们将代码公开可用。