We present a novel generative 3D modeling system, coined CraftsMan, which can
generate high-fidelity 3D geometries with highly varied shapes, regular mesh
topologies, and detailed surfaces, and, notably, allows for refining the
geometry in an interactive manner. Despite the significant advancements in 3D
generation, existing methods still struggle with lengthy optimization
processes, irregular mesh topologies, noisy surfaces, and difficulties in
accommodating user edits, consequently impeding their widespread adoption and
implementation in 3D modeling software. Our work is inspired by the craftsman,
who usually roughs out the holistic figure of the work first and elaborates the
surface details subsequently. Specifically, we employ a 3D native diffusion
model, which operates on latent space learned from latent set-based 3D
representations, to generate coarse geometries with regular mesh topology in
seconds. In particular, this process takes as input a text prompt or a
reference image and leverages a powerful multi-view (MV) diffusion model to
generate multiple views of the coarse geometry, which are fed into our
MV-conditioned 3D diffusion model for generating the 3D geometry, significantly
improving robustness and generalizability. Following that, a normal-based
geometry refiner is used to significantly enhance the surface details. This
refinement can be performed automatically, or interactively with user-supplied
edits. Extensive experiments demonstrate that our method achieves high efficacy
in producing superior-quality 3D assets compared to existing methods. HomePage:
this https URL, Code: this https URL

我们提出了一种新颖的生成式 3D 建模系统，被称为 CraftsMan，它能够以高度多样化的形状、规则的网格拓扑和详细的表面生成高保真度的 3D 几何模型，并且能够交互式地进行几何调整。