Despite recent advancements in the Large Reconstruction Model (LRM)
demonstrating impressive results, when extending its input from single image to
multiple images, it exhibits inefficiencies, subpar geometric and texture
quality, as well as slower convergence speed than expected.
It is attributed to that, LRM formulates 3D reconstruction as a naive
images-to-3D translation problem, ignoring the strong 3D coherence among the
input images. In this paper, we propose a Multi-view Large Reconstruction Model
(M-LRM) designed to efficiently reconstruct high-quality 3D shapes from
multi-views in a 3D-aware manner. Specifically, we introduce a multi-view
consistent cross-attention scheme to enable M-LRM to accurately query
information from the input images. Moreover, we employ the 3D priors of the
input multi-view images to initialize the tri-plane tokens. Compared to LRM,
the proposed M-LRM can produce a tri-plane NeRF with $128 \times 128$
resolution and generate 3D shapes of high fidelity. Experimental studies
demonstrate that our model achieves a significant performance gain and faster
training convergence than LRM. Project page: this https URL

通过引入跨视图一致的交叉注意力机制和利用三维先验信息，提出了一种高效的多视角大型重建模型 (M-LRM)，能够从多视图中以三维感知的方式高质量地重建三维形状，并且相比于 LRM 能够生成分辨率为 $128 	imes 128$ 的三平面 NeRF，具有更高的保真度和训练收敛速度。

M-LRM: 多视角大重建模型

M-LRM: Multi-view Large Reconstruction Model

We propose the first Large Reconstruction Model (LRM) that predicts the 3D
model of an object from a single input image within just 5 seconds. In contrast
to many previous methods that are trained on small-scale datasets such as
ShapeNet in a category-specific fashion, LRM adopts a highly scalable
transformer-based architecture with 500 million learnable parameters to
directly predict a neural radiance field (NeRF) from the input image. We train
our model in an end-to-end manner on massive multi-view data containing around
1 million objects, including both synthetic renderings from Objaverse and real
captures from MVImgNet. This combination of a high-capacity model and
large-scale training data empowers our model to be highly generalizable and
produce high-quality 3D reconstructions from various testing inputs including
real-world in-the-wild captures and images from generative models. Video demos
and interactable 3D meshes can be found on this website:
this https URL

我们提出了第一个大规模重建模型 (LRM)，能够在仅 5 秒内从单个输入图像预测对象的 3D 模型。与许多以类别为基础在小规模数据集（如 ShapeNet）上训练的先前方法不同，LRM 采用一个高度可扩展的基于 transformer 的架构，具有 5 亿个可学习参数，能够直接从输入图像预测神经辐射场（NeRF）。我们以端到端的方式在包含大约 100 万个对象的海量多视图数据上训练我们的模型，包括 Objaverse 的合成渲染和 MVImgNet 的真实采集数据。这种高容量模型和大规模的训练数据的组合使得我们的模型具有很强的通用性，并能够从各种测试输入中生成高质量的 3D 重建结果，包括真实世界中的野外捕捉和生成模型的图像。可在此网站找到视频演示和可交互的 3D 网格：[https://this_URL]。