Large Reconstruction Models have made significant strides in the realm of
automated 3D content generation from single or multiple input images. Despite
their success, these models often produce 3D meshes with geometric
inaccuracies, stemming from the inherent challenges of deducing 3D shapes
solely from image data. In this work, we introduce a novel framework, the Large
Image and Point Cloud Alignment Model (LAM3D), which utilizes 3D point cloud
data to enhance the fidelity of generated 3D meshes. Our methodology begins
with the development of a point-cloud-based network that effectively generates
precise and meaningful latent tri-planes, laying the groundwork for accurate 3D
mesh reconstruction. Building upon this, our Image-Point-Cloud Feature
Alignment technique processes a single input image, aligning to the latent
tri-planes to imbue image features with robust 3D information. This process not
only enriches the image features but also facilitates the production of
high-fidelity 3D meshes without the need for multi-view input, significantly
reducing geometric distortions. Our approach achieves state-of-the-art
high-fidelity 3D mesh reconstruction from a single image in just 6 seconds, and
experiments on various datasets demonstrate its effectiveness.

大规模重建模型在自动化三维内容生成方面取得了重大突破。然而，这些模型常常产生具有几何不精确性的三维网格，这是由于仅从图像数据推断三维形状的固有挑战所引起的。在本文中，我们介绍了一种新颖的框架，即大型图像与点云对齐模型（LAM3D），它利用三维点云数据增强了生成的三维网格的保真度。我们的方法从开发基于点云的网络开始，有效地生成精确而有意义的潜在三平面，为准确的三维网格重建奠定了基础。在此基础上，我们的图像 - 点云特征对齐技术通过处理单个输入图像，将其与潜在三平面对齐，使图像特征具备强大的三维信息。这个过程不仅丰富了图像特征，还有助于产生高保真度的三维网格，而无需多视图输入，大大降低了几何变形。我们的方法在仅 6 秒内实现了单个图像的最先进高保真度三维网格重建，并在各种数据集上的实验证明了它的有效性。