Recent breakthroughs in text-to-image synthesis have been driven by diffusion
models trained on billions of image-text pairs. Adapting this approach to 3D
synthesis would require large-scale datasets of labeled 3D data and efficient
architectures for denoising 3D data, neither of which currently exist. In this
work, we circumvent these limitations by using a pretrained 2D text-to-image
diffusion model to perform text-to-3D synthesis. We introduce a loss based on
probability density distillation that enables the use of a 2D diffusion model
as a prior for optimization of a parametric image generator. Using this loss in
a DeepDream-like procedure, we optimize a randomly-initialized 3D model (a
Neural Radiance Field, or NeRF) via gradient descent such that its 2D
renderings from random angles achieve a low loss. The resulting 3D model of the
given text can be viewed from any angle, relit by arbitrary illumination, or
composited into any 3D environment. Our approach requires no 3D training data
and no modifications to the image diffusion model, demonstrating the
effectiveness of pretrained image diffusion models as priors.

本文提出了一种使用基于文本转图像的扩散模型进行文本到 3D 合成的方法，该方法绕过了需要大规模标记的 3D 数据集和能够去噪的 3D 数据的限制，将 2D 的扩散模型作为先验，通过梯度下降优化 3D 模型（Neural Radiance Field），并使用概率密度蒸馏引入的损失函数将 2D 扩散模型与 3D 模型相结合。这种方法不需要 3D 训练数据，也不需要修改图像扩散模型，证明了使用预训练的图像扩散模型作为先验的有效性。

DreamFusion: 使用 2D 扩散进行文本到 3D 转换

DreamFusion: Text-to-3D using 2D Diffusion

This paper proposes an effective probability density distillation (PDD)
algorithm for WaveNet-based parallel waveform generation (PWG) systems.
Recently proposed teacher-student frameworks in the PWG system have
successfully achieved a real-time generation of speech signals. However, the
difficulties optimizing the PDD criteria without auxiliary losses result in
quality degradation of synthesized speech. To generate more natural speech
signals within the teacher-student framework, we propose a novel optimization
criterion based on generative adversarial networks (GANs). In the proposed
method, the inverse autoregressive flow-based student model is incorporated as
a generator in the GAN framework, and jointly optimized by the PDD mechanism
with the proposed adversarial learning method. As this process encourages the
student to model the distribution of realistic speech waveform, the perceptual
quality of the synthesized speech becomes much more natural. Our experimental
results verify that the PWG systems with the proposed method outperform both
those using conventional approaches, and also autoregressive generation systems
with a well-trained teacher WaveNet.

本文提供一种基于众多方法的构思，用于波形生成模型中的概率密度蒸馏算法，以产生更加自然的语音信号。

使用生成对抗网络进行概率密度蒸馏，用于高质量并行波形生成

Probability density distillation with generative adversarial networks  for high-quality parallel waveform generation

The recently-developed WaveNet architecture is the current state of the art
in realistic speech synthesis, consistently rated as more natural sounding for
many different languages than any previous system. However, because WaveNet
relies on sequential generation of one audio sample at a time, it is poorly
suited to today's massively parallel computers, and therefore hard to deploy in
a real-time production setting. This paper introduces Probability Density
Distillation, a new method for training a parallel feed-forward network from a
trained WaveNet with no significant difference in quality. The resulting system
is capable of generating high-fidelity speech samples at more than 20 times
faster than real-time, and is deployed online by Google Assistant, including
serving multiple English and Japanese voices.

采用概率密度蒸馏技术，从 WaveNet 模型中训练一个并行前向网络，实现语音样本的超过 20 倍实时生成速度，目前已部署在 Google 助手中支持多种不同英文和日语语音。