Composite Functional Gradient Learning of Generative Adversarial Models
share
Generative adversarial networks (GAN) have become popular for generating data that mimic observations by learning a suitable variable transformation from a random variable. However, empirically, GAN is known to suffer from instability. Also, the theory provided based on the minimax optimization formulation of GAN cannot explain the widely-used practical procedure that uses the so-called logd trick. This paper provides a different theoretical foundation for generative adversarial methods which does not rely on the minimax formulation. We show that with a strong discriminator, it is possible to learn a good variable transformation via functional gradient learning, which updates the functional definition of a generator model, instead of updating only the model parameters as in GAN. The theory guarantees that the learned generator improves the KL-divergence between the probability distributions of real data and generated data after each functional gradient step, until the KL-divergence converges to zero. This new point of view leads to enhanced stable procedures for training generative models that can utilize arbitrary learning algorithms. It also gives a new theoretical insight into the original GAN procedure both with and without the logd trick. Empirical results are shown on image generation to illustrate the effectiveness of our new method.
READ FULL TEXT