We propose a simple yet effective pipeline for stylizing a 3D scene,
harnessing the power of 2D image diffusion models. Given a NeRF model
reconstructed from a set of multi-view images, we perform 3D style transfer by
refining the source NeRF model using stylized images generated by a
style-aligned image-to-image diffusion model. Given a target style prompt, we
first generate perceptually similar multi-view images by leveraging a
depth-conditioned diffusion model with an attention-sharing mechanism. Next,
based on the stylized multi-view images, we propose to guide the style transfer
process with the sliced Wasserstein loss based on the feature maps extracted
from a pre-trained CNN model. Our pipeline consists of decoupled steps,
allowing users to test various prompt ideas and preview the stylized 3D result
before proceeding to the NeRF fine-tuning stage. We demonstrate that our method
can transfer diverse artistic styles to real-world 3D scenes with competitive
quality.

通过使用 2D 图像扩散模型来改良 3D 场景，我们提出了一种简单但有效的流程；通过细化源 NeRF 模型以及使用风格化图像生成的样式对齐的图像扩散模型，实现 3D 风格转移。

Style-NeRF2NeRF: 从样式对齐的多视图图像中进行三维样式转移

Style-NeRF2NeRF: 3D Style Transfer From Style-Aligned Multi-View Images

Neural Radiance Field (NeRF) models have gained significant attention in the
computer vision community in the recent past with state-of-the-art visual
quality and produced impressive demonstrations. Since then, technopreneurs have
sought to leverage NeRF models into a profitable business. Therefore, NeRF
models make it worth the risk of plagiarizers illegally copying,
re-distributing, or misusing those models. This paper proposes a comprehensive
intellectual property (IP) protection framework for the NeRF model in both
black-box and white-box settings, namely IPR-NeRF. In the black-box setting, a
diffusion-based solution is introduced to embed and extract the watermark via a
two-stage optimization process. In the white-box setting, a designated digital
signature is embedded into the weights of the NeRF model by adopting the sign
loss objective. Our extensive experiments demonstrate that not only does our
approach maintain the fidelity (\ie, the rendering quality) of IPR-NeRF models,
but it is also robust against both ambiguity and removal attacks compared to
prior arts.

该研究提出了一种全面的知识产权 (IP) 保护框架，名为 IPR-NeRF，用于保护 Neural Radiance Field (NeRF) 模型的版权，其中包括黑盒模式和白盒模式两种设置。

IPR-NeRF：所有权验证与神经亮度场相遇

IPR-NeRF: Ownership Verification meets Neural Radiance Field

The increasing demand for high-quality 3D content creation has motivated the
development of automated methods for creating 3D object models from a single
image and/or from a text prompt. However, the reconstructed 3D objects using
state-of-the-art image-to-3D methods still exhibit low correspondence to the
given image and low multi-view consistency. Recent state-of-the-art text-to-3D
methods are also limited, yielding 3D samples with low diversity per prompt
with long synthesis time. To address these challenges, we propose DITTO-NeRF, a
novel pipeline to generate a high-quality 3D NeRF model from a text prompt or a
single image. Our DITTO-NeRF consists of constructing high-quality partial 3D
object for limited in-boundary (IB) angles using the given or text-generated 2D
image from the frontal view and then iteratively reconstructing the remaining
3D NeRF using inpainting latent diffusion model. We propose progressive 3D
object reconstruction schemes in terms of scales (low to high resolution),
angles (IB angles initially to outer-boundary (OB) later), and masks (object to
background boundary) in our DITTO-NeRF so that high-quality information on IB
can be propagated into OB. Our DITTO-NeRF outperforms state-of-the-art methods
in terms of fidelity and diversity qualitatively and quantitatively with much
faster training times than prior arts on image/text-to-3D such as DreamFusion,
and NeuralLift-360.

提出了一种名为 DITTO-NeRF 的新型流水线，用于从文本提示或单个图像生成高质量的 3D NeRF 模型，并通过逐步重建规模、角度和遮罩等方案来解决 Image-to-3D 方法的低对应性和多视图一致性的问题，在保留微小边界角度信息的情况下，实现对 NeRF 模型的高保真度和多样性重建。