Most 3D generation research focuses on up-projecting 2D foundation models
into the 3D space, either by minimizing 2D Score Distillation Sampling (SDS)
loss or fine-tuning on multi-view datasets. Without explicit 3D priors, these
methods often lead to geometric anomalies and multi-view inconsistency.
Recently, researchers have attempted to improve the genuineness of 3D objects
by directly training on 3D datasets, albeit at the cost of low-quality texture
generation due to the limited texture diversity in 3D datasets. To harness the
advantages of both approaches, we propose Bidirectional Diffusion(BiDiff), a
unified framework that incorporates both a 3D and a 2D diffusion process, to
preserve both 3D fidelity and 2D texture richness, respectively. Moreover, as a
simple combination may yield inconsistent generation results, we further bridge
them with novel bidirectional guidance. In addition, our method can be used as
an initialization of optimization-based models to further improve the quality
of 3D model and efficiency of optimization, reducing the generation process
from 3.4 hours to 20 minutes. Experimental results have shown that our model
achieves high-quality, diverse, and scalable 3D generation. Project website:
this https URL

以 Bidirectional Diffusion（BiDiff）为框架，同时整合 3D 和 2D 扩散过程，既保持了 3D 的真实性，又保留了 2D 纹理的丰富性，通过新颖的双向引导进一步提高一致性，将生成的过程从 3.4 小时减少到 20 分钟，以达到高质量、多样性和可伸缩性的 3D 生成。

使用双向扩散结合 2D 和 3D 先验的文本至 3D 生成

Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D  priors

Blind super-resolution (BSR) methods based on high-resolution (HR)
reconstruction codebooks have achieved promising results in recent years.
However, we find that a codebook based on HR reconstruction may not effectively
capture the complex correlations between low-resolution (LR) and HR images. In
detail, multiple HR images may produce similar LR versions due to complex blind
degradations, causing the HR-dependent only codebooks having limited texture
diversity when faced with confusing LR inputs. To alleviate this problem, we
propose the Rich Texture-aware Codebook-based Network (RTCNet), which consists
of the Degradation-robust Texture Prior Module (DTPM) and the Patch-aware
Texture Prior Module (PTPM). DTPM effectively mines the cross-resolution
correlation of textures between LR and HR images by exploiting the
cross-resolution correspondence of textures. PTPM uses patch-wise semantic
pre-training to correct the misperception of texture similarity in the
high-level semantic regularization. By taking advantage of this, RTCNet
effectively gets rid of the misalignment of confusing textures between HR and
LR in the BSR scenarios. Experiments show that RTCNet outperforms
state-of-the-art methods on various benchmarks by up to 0.16 ~ 0.46dB.

基于高分辨率重建码书的盲超分辨率方法在最近几年取得了令人满意的结果。然而，我们发现，基于高分辨率重建的码书可能无法有效捕捉低分辨率和高分辨率图像之间的复杂相关性。为了解决这个问题，我们提出了 Rich Texture-aware Codebook-based Network (RTCNet)，通过利用纹理的跨分辨率对应关系和对纹理相似度的高层语义正则化进行修正，从而有效消除了 BSR 场景中高分辨率和低分辨率之间混淆纹理的不对齐问题。实验表明，RTCNet 在各种基准测试中优于现有方法，提高了 0.16~0.46dB。