Many generative models have to combat $\textit{missing modes}$. The
conventional wisdom to this end is by reducing through training a statistical
distance (such as $f$-divergence) between the generated distribution and
provided data distribution. But this is more of a heuristic than a guarantee.
The statistical distance measures a $\textit{global}$, but not
$\textit{local}$, similarity between two distributions. Even if it is small, it
does not imply a plausible mode coverage. Rethinking this problem from a
game-theoretic perspective, we show that a complete mode coverage is firmly
attainable. If a generative model can approximate a data distribution
moderately well under a global statistical distance measure, then we will be
able to find a mixture of generators that collectively covers $\textit{every}$
data point and thus $\textit{every}$ mode, with a lower-bounded generation
probability. Constructing the generator mixture has a connection to the
multiplicative weights update rule, upon which we propose our algorithm. We
prove that our algorithm guarantees complete mode coverage. And our experiments
on real and synthetic datasets confirm better mode coverage over recent
approaches, ones that also use generator mixtures but rely on global
statistical distances.

通过游戏论的角度重新思考模态缺失问题，提出了一种基于乘性权重更新规则的生成模型算法，能够通过生成器混合覆盖每个数据点和每个模式。

重新思考生成模型覆盖度：点对点保证方法

Rethinking Generative Mode Coverage: A Pointwise Guaranteed Approach

Generative adversarial networks (GANs) while being very versatile in
realistic image synthesis, still are sensitive to the input distribution. Given
a set of data that has an imbalance in the distribution, the networks are
susceptible to missing modes and not capturing the data distribution. While
various methods have been tried to improve training of GANs, these have not
addressed the challenges of covering the full data distribution. Specifically,
a generator is not penalized for missing a mode. We show that these are
therefore still susceptible to not capturing the full data distribution.
In this paper, we propose a simple approach that combines an encoder based
objective with novel loss functions for generator and discriminator that
improves the solution in terms of capturing missing modes. We validate that the
proposed method results in substantial improvements through its detailed
analysis on toy and real datasets. The quantitative and qualitative results
demonstrate that the proposed method improves the solution for the problem of
missing modes and improves training of GANs.

本文提出了一种新的简单方法，将基于编码器的目标函数与新颖的损失函数相结合，以在丢失模式问题上改进生成对抗网络（GAN）的训练，其中详细分析了玩具和真实数据集的量化和定性结果，证明了该方法在解决缺失模式问题和提高 GAN 训练方面的有效性。

没有模式被遗漏：使用 GAN 有效捕获数据分布

No Modes left behind: Capturing the data distribution effectively using  GANs

Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) are an
effective method for training generative models of complex data such as natural
images. However, they are notoriously hard to train and can suffer from the
problem of missing modes where the model is not able to produce examples in
certain regions of the space. We propose an iterative procedure, called AdaGAN,
where at every step we add a new component into a mixture model by running a
GAN algorithm on a reweighted sample. This is inspired by boosting algorithms,
where many potentially weak individual predictors are greedily aggregated to
form a strong composite predictor. We prove that such an incremental procedure
leads to convergence to the true distribution in a finite number of steps if
each step is optimal, and convergence at an exponential rate otherwise. We also
illustrate experimentally that this procedure addresses the problem of missing
modes.

该研究使用增量算法 AdaGAN 训练 Generative Adversarial Networks（GAN）来解决 GAN 模型中出现的 missing modes 问题，并且证明了当每一步是最优的时候，这种增量方法可以在有限步数内收敛到真实分布，否则以指数速度收敛。

AdaGAN：提升生成式模型

AdaGAN: Boosting Generative Models

Although Generative Adversarial Networks achieve state-of-the-art results on
a variety of generative tasks, they are regarded as highly unstable and prone
to miss modes. We argue that these bad behaviors of GANs are due to the very
particular functional shape of the trained discriminators in high dimensional
spaces, which can easily make training stuck or push probability mass in the
wrong direction, towards that of higher concentration than that of the data
generating distribution. We introduce several ways of regularizing the
objective, which can dramatically stabilize the training of GAN models. We also
show that our regularizers can help the fair distribution of probability mass
across the modes of the data generating distribution, during the early phases
of training and thus providing a unified solution to the missing modes problem.

本研究论文证明生成式对抗网络的不稳定性和漏模现象源自鉴别器在高位空间内的特殊功能形状，提出了多种正则化目标的解决方案从而实现生成模型的稳定训练并解决数据生成分布的漏模问题。