Open-domain 3D object synthesis has been lagging behind image synthesis due
to limited data and higher computational complexity. To bridge this gap, recent
works have investigated multi-view diffusion but often fall short in either 3D
consistency, visual quality, or efficiency. This paper proposes MVEdit, which
functions as a 3D counterpart of SDEdit, employing ancestral sampling to
jointly denoise multi-view images and output high-quality textured meshes.
Built on off-the-shelf 2D diffusion models, MVEdit achieves 3D consistency
through a training-free 3D Adapter, which lifts the 2D views of the last
timestep into a coherent 3D representation, then conditions the 2D views of the
next timestep using rendered views, without uncompromising visual quality. With
an inference time of only 2-5 minutes, this framework achieves better trade-off
between quality and speed than score distillation. MVEdit is highly versatile
and extendable, with a wide range of applications including text/image-to-3D
generation, 3D-to-3D editing, and high-quality texture synthesis. In
particular, evaluations demonstrate state-of-the-art performance in both
image-to-3D and text-guided texture generation tasks. Additionally, we
introduce a method for fine-tuning 2D latent diffusion models on small 3D
datasets with limited resources, enabling fast low-resolution text-to-3D
initialization.

MVEdit 是一个高度灵活和可扩展的框架，结合了多视角扩散和祖先采样技术，在只需 2-5 分钟的推理时间内实现了在质量和速度之间更好的平衡，对于开放领域的 3D 对象合成，多视角扩散以及高质量贴图生成等应用具有具有最先进性能的评估结果。

使用受控的多视图编辑的通用 3D 扩散适配器

Generic 3D Diffusion Adapter Using Controlled Multi-View Editing

This paper introduces a novel approach to synthesize texture to dress up a
given 3D object, given a text prompt. Based on the pretrained text-to-image
(T2I) diffusion model, existing methods usually employ a project-and-inpaint
approach, in which a view of the given object is first generated and warped to
another view for inpainting. But it tends to generate inconsistent texture due
to the asynchronous diffusion of multiple views. We believe such asynchronous
diffusion and insufficient information sharing among views are the root causes
of the inconsistent artifact. In this paper, we propose a synchronized
multi-view diffusion approach that allows the diffusion processes from
different views to reach a consensus of the generated content early in the
process, and hence ensures the texture consistency. To synchronize the
diffusion, we share the denoised content among different views in each
denoising step, specifically blending the latent content in the texture domain
from views with overlap. Our method demonstrates superior performance in
generating consistent, seamless, highly detailed textures, comparing to
state-of-the-art methods.

介绍一种新颖的方法用于合成给定 3D 对象的纹理，通过预训练的文本到图像扩散模型，提出了一种同步多视点扩散方法，早期生成的纹理内容达成一致，确保纹理的一致性。

同步多视角扩散下的文本引导纹理

Text-Guided Texturing by Synchronized Multi-View Diffusion

We propose \textbf{DMV3D}, a novel 3D generation approach that uses a
transformer-based 3D large reconstruction model to denoise multi-view
diffusion. Our reconstruction model incorporates a triplane NeRF representation
and can denoise noisy multi-view images via NeRF reconstruction and rendering,
achieving single-stage 3D generation in $\sim$30s on single A100 GPU. We train
\textbf{DMV3D} on large-scale multi-view image datasets of highly diverse
objects using only image reconstruction losses, without accessing 3D assets. We
demonstrate state-of-the-art results for the single-image reconstruction
problem where probabilistic modeling of unseen object parts is required for
generating diverse reconstructions with sharp textures. We also show
high-quality text-to-3D generation results outperforming previous 3D diffusion
models. Our project website is at: this https URL .

我们提出了一种新颖的 3D 生成方法 DMV3D，该方法使用基于变压器的 3D 大型重建模型对多视差扩散进行去噪。我们的重建模型采用了三平面 NeRF 表示，并且可以通过 NeRF 重建和渲染对嘈杂的多视差图像进行去噪，能在单个 A100 GPU 上实现约 30 秒的单阶段 3D 生成。我们使用大规模多视差图像数据集训练 DMV3D，仅使用图像重建损失，而不访问 3D 资产。我们展示了在需要对未见过的物体部分进行概率建模以生成具有清晰纹理的多样重建的单图像重建问题上的最新成果，以及优于以往的 3D 扩散模型的高质量文本到 3D 生成结果。我们的项目网站位于此 https URL。