Implicit Manifold Learning on Generative Adversarial Networks
share
This paper raises an implicit manifold learning perspective in Generative Adversarial Networks (GANs), by studying how the support of the learned distribution, modelled as a submanifold M_θ, perfectly match with M_r, the support of the real data distribution. We show that optimizing Jensen-Shannon divergence forces M_θ to perfectly match with M_r, while optimizing Wasserstein distance does not. On the other hand, by comparing the gradients of the Jensen-Shannon divergence and the Wasserstein distances (W_1 and W_2^2) in their primal forms, we conjecture that Wasserstein W_2^2 may enjoy desirable properties such as reduced mode collapse. It is therefore interesting to design new distances that inherit the best from both distances.
READ FULL TEXT