Generative adversarial networks (GAN) have become popular for generating data that mimic observations by learning a suitable variable transformation from a random variable. However, empirically, GAN is known to suffer from instability. Also, the theory provided based on the minimax optimization formulation of GAN cannot explain the widely-used practical procedure that uses the so-called logd trick. This paper provides a different theoretical foundation for generative adversarial methods which does not rely on the minimax formulation. We show that with a strong discriminator, it is possible to learn a good variable transformation via functional gradient learning, which updates the functional definition of a generator model, instead of updating only the model parameters as in GAN. The theory guarantees that the learned generator improves the KL-divergence between the probability distributions of real data and generated data after each functional gradient step, until the KL-divergence converges to zero. This new point of view leads to enhanced stable procedures for training generative models that can utilize arbitrary learning algorithms. It also gives a new theoretical insight into the original GAN procedure both with and without the logd trick. Empirical results are shown on image generation to illustrate the effectiveness of our new method.

本文提出了一种不依赖于传统极小-极大公式的生成式对抗方法理论，并展示了当存在强的辨别器时，通过每个功能性梯度步骤，可以学到一个好的生成器，使得真实数据和生成数据的分布的KL散度改善，直到收敛于零，并基于该理论，提出了一种新的稳定的生成式对抗方法，同时提供了从这个新的视角对原始GAN的理论洞见，最终，针对图像生成的实验展示了我们的新方法的有效性。

生成对抗模型的组合功能梯度学习