Recovering a textured 3D mesh from a monocular image is highly challenging,
particularly for in-the-wild objects that lack 3D ground truths. In this work,
we present MeshInversion, a novel framework to improve the reconstruction by
exploiting the generative prior of a 3D GAN pre-trained for 3D textured mesh
synthesis. Reconstruction is achieved by searching for a latent space in the 3D
GAN that best resembles the target mesh in accordance with the single view
observation. Since the pre-trained GAN encapsulates rich 3D semantics in terms
of mesh geometry and texture, searching within the GAN manifold thus naturally
regularizes the realness and fidelity of the reconstruction. Importantly, such
regularization is directly applied in the 3D space, providing crucial guidance
of mesh parts that are unobserved in the 2D space. Experiments on standard
benchmarks show that our framework obtains faithful 3D reconstructions with
consistent geometry and texture across both observed and unobserved parts.
Moreover, it generalizes well to meshes that are less commonly seen, such as
the extended articulation of deformable objects. Code is released at
this https URL

通过利用预先训练的 3D GAN 生成先验来改善重建结果，本文提出了一种名为 MeshInversion 的新型框架，它在单视角观察下通过搜索最相似目标网格的 3D GAN 的潜在空间来实现重建，从而在 3D 空间中直接应用正则化，提供了未观测的网格部位的关键指导，实现了出色的 3D 重建。