GANs May Have No Nash Equilibria

by   Farzan Farnia, et al.

Generative adversarial networks (GANs) represent a zero-sum game between two machine players, a generator and a discriminator, designed to learn the distribution of data. While GANs have achieved state-of-the-art performance in several benchmark learning tasks, GAN minimax optimization still poses great theoretical and empirical challenges. GANs trained using first-order optimization methods commonly fail to converge to a stable solution where the players cannot improve their objective, i.e., the Nash equilibrium of the underlying game. Such issues raise the question of the existence of Nash equilibrium solutions in the GAN zero-sum game. In this work, we show through several theoretical and numerical results that indeed GAN zero-sum games may not have any local Nash equilibria. To characterize an equilibrium notion applicable to GANs, we consider the equilibrium of a new zero-sum game with an objective function given by a proximal operator applied to the original objective, a solution we call the proximal equilibrium. Unlike the Nash equilibrium, the proximal equilibrium captures the sequential nature of GANs, in which the generator moves first followed by the discriminator. We prove that the optimal generative model in Wasserstein GAN problems provides a proximal equilibrium. Inspired by these results, we propose a new approach, which we call proximal training, for solving GAN problems. We discuss several numerical experiments demonstrating the existence of proximal equilibrium solutions in GAN minimax problems.


page 19

page 20

page 21

page 24

page 25

page 26

page 28

page 29


Beyond Local Nash Equilibria for Adversarial Networks

Save for some special cases, current training methods for Generative Adv...

Characterizing GAN Convergence Through Proximal Duality Gap

Despite the accomplishments of Generative Adversarial Networks (GANs) in...

DO-GAN: A Double Oracle Framework for Generative Adversarial Networks

In this paper, we propose a new approach to train Generative Adversarial...

Coulomb GANs: Provably Optimal Nash Equilibria via Potential Fields

Generative adversarial networks (GANs) evolved into one of the most succ...

Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step

Generative adversarial networks (GANs) are a family of generative models...

On the Nash equilibrium of moment-matching GANs for stationary Gaussian processes

Generative Adversarial Networks (GANs) learn an implicit generative mode...

GANGs: Generative Adversarial Network Games

Generative Adversarial Networks (GAN) have become one of the most succes...

Please sign up or login with your details

Forgot password? Click here to reset