We study the problem of training a flow-based generative model, parametrized
by a two-layer autoencoder, to sample from a high-dimensional Gaussian mixture.
We provide a sharp end-to-end analysis of the problem. First, we provide a
tight closed-form characterization of the learnt velocity field, when
parametrized by a shallow denoising auto-encoder trained on a finite number $n$
of samples from the target distribution. Building on this analysis, we provide
a sharp description of the corresponding generative flow, which pushes the base
Gaussian density forward to an approximation of the target density. In
particular, we provide closed-form formulae for the distance between the mean
of the generated mixture and the mean of the target mixture, which we show
decays as $\Theta_n(\frac{1}{n})$. Finally, this rate is shown to be in fact
Bayes-optimal.

使用两层自编码器参数化的流式生成模型，通过对目标分布从有限数量 n 的样本进行训练，提供了问题的尖锐端到端分析，包括对学习速度场的紧密闭合形式表征以及对相应生成流的尖锐描述，最终证明该速度场的收敛速度是贝叶斯最优的。

从有限样本复杂性中学习基于流的生成模型的分析

Analysis of learning a flow-based generative model from limited sample  complexity

In this work, we propose WaveFlow, a small-footprint generative flow for raw
audio, which is directly trained with maximum likelihood. It handles the
long-range structure of 1-D waveform with a dilated 2-D convolutional
architecture, while modeling the local variations using expressive
autoregressive functions. WaveFlow provides a unified view of likelihood-based
models for 1-D data, including WaveNet and WaveGlow as special cases. It
generates high-fidelity speech as WaveNet, while synthesizing several orders of
magnitude faster as it only requires a few sequential steps to generate very
long waveforms with hundreds of thousands of time-steps. Furthermore, it can
significantly reduce the likelihood gap that has existed between autoregressive
models and flow-based models for efficient synthesis. Finally, our
small-footprint WaveFlow has only 5.91M parameters, which is 15$\times$ smaller
than WaveGlow. It can generate 22.05 kHz high-fidelity audio 42.6$\times$
faster than real-time (at a rate of 939.3 kHz) on a V100 GPU without engineered
inference kernels.

本文介绍了一种小型的 WaveFlow 产生式流，可以训练原始音频并合成高保真语音，只需几个步骤即可生成成千上万个时间步长的波形，并具有比 WaveGlow 小 15 倍的参数和 42.6 倍的音频合成速度。

WaveFlow: 面向原始音频的紧凑流模型

WaveFlow: A Compact Flow-based Model for Raw Audio

Most sequence-to-sequence (seq2seq) models are autoregressive; they generate
each token by conditioning on previously generated tokens. In contrast,
non-autoregressive seq2seq models generate all tokens in one pass, which leads
to increased efficiency through parallel processing on hardware such as GPUs.
However, directly modeling the joint distribution of all tokens simultaneously
is challenging, and even with increasingly complex model structures accuracy
lags significantly behind autoregressive models. In this paper, we propose a
simple, efficient, and effective model for non-autoregressive sequence
generation using latent variable models. Specifically, we turn to generative
flow, an elegant technique to model complex distributions using neural
networks, and design several layers of flow tailored for modeling the
conditional density of sequential latent variables. We evaluate this model on
three neural machine translation (NMT) benchmark datasets, achieving comparable
performance with state-of-the-art non-autoregressive NMT models and almost
constant decoding time w.r.t the sequence length.

提出一种使用生成流的非自回归序列生成模型，以提高非自回归序列生成的效率。在三个神经机器翻译基准数据集上进行了评估，与最先进的非自回归神经机器翻译模型性能相当且几乎不随序列长度而变化。

FlowSeq：基于生成流的非自回归条件序列生成

FlowSeq: Non-Autoregressive Conditional Sequence Generation with  Generative Flow

Traditional structured prediction models try to learn the conditional
likelihood, i.e., p(y|x), to capture the relationship between the structured
output y and the input features x. For many models, computing the likelihood is
intractable. These models are therefore hard to train, requiring the use of
surrogate objectives or variational inference to approximate likelihood. In
this paper, we propose conditional Glow (c-Glow), a conditional generative flow
for structured output learning. C-Glow benefits from the ability of flow-based
models to compute p(y|x) exactly and efficiently. Learning with c-Glow does not
require a surrogate objective or performing inference during training. Once
trained, we can directly and efficiently generate conditional samples. We
develop a sample-based prediction method, which can use this advantage to do
efficient and effective inference. In our experiments, we test c-Glow on five
different tasks. C-Glow outperforms the state-of-the-art baselines in some
tasks and predicts comparable outputs in the other tasks. The results show that
c-Glow is versatile and is applicable to many different structured prediction
problems.

本文提出了一种叫做 c-Glow 的条件生成流算法，用于结构化输出学习，该算法能够准确有效地计算 p（y | x），不需要使用代理目标或在训练期间执行推理。