Stackelberg GAN: Towards Provable Minimax Equilibrium via Multi-Generator Architectures

by   Hongyang Zhang, et al.
Petuum, Inc.
Carnegie Mellon University
berkeley college

We study the problem of alleviating the instability issue in the GAN training procedure via new architecture design. The discrepancy between the minimax and maximin objective values could serve as a proxy for the difficulties that the alternating gradient descent encounters in the optimization of GANs. In this work, we give new results on the benefits of multi-generator architecture of GANs. We show that the minimax gap shrinks to ϵ as the number of generators increases with rate O(1/ϵ). This improves over the best-known result of O(1/ϵ^2). At the core of our techniques is a novel application of Shapley-Folkman lemma to the generic minimax problem, where in the literature the technique was only known to work when the objective function is restricted to the Lagrangian function of a constraint optimization problem. Our proposed Stackelberg GAN performs well experimentally in both synthetic and real-world datasets, improving Fréchet Inception Distance by 14.61% over the previous multi-generator GANs on the benchmark datasets.


page 2

page 8

page 9

page 10

page 24

page 25

page 26

page 27


Deconstructing Generative Adversarial Networks

We deconstruct the performance of GANs into three components: 1. Formu...

Training Generative Adversarial Networks via Primal-Dual Subgradient Methods: A Lagrangian Perspective on GAN

We relate the minimax game of generative adversarial networks (GANs) to ...

Solve Minimax Optimization by Anderson Acceleration

Many modern machine learning algorithms such as generative adversarial n...

An Online Learning Approach to Generative Adversarial Networks

We consider the problem of training generative models with a Generative ...

Is Generator Conditioning Causally Related to GAN Performance?

Recent work (Pennington et al, 2017) suggests that controlling the entir...

Faster Single-loop Algorithms for Minimax Optimization without Strong Concavity

Gradient descent ascent (GDA), the simplest single-loop algorithm for no...

Memory-Efficient Sampling for Minimax Distance Measures

Minimax distance measure extracts the underlying patterns and manifolds ...

Code Repositories


Codes for Stackelberg GAN

view repo

Please sign up or login with your details

Forgot password? Click here to reset