CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training

by   Jianmin Bao, et al.

We present variational generative adversarial networks, a general learning framework that combines a variational auto-encoder with a generative adversarial network, for synthesizing images in fine-grained categories, such as faces of a specific person or objects in a category. Our approach models an image as a composition of label and latent attributes in a probabilistic model. By varying the fine-grained category label fed into the resulting generative model, we can generate images in a specific category with randomly drawn values on a latent attribute vector. Our approach has two novel aspects. First, we adopt a cross entropy loss for the discriminative and classifier network, but a mean discrepancy objective for the generative network. This kind of asymmetric loss function makes the GAN training more stable. Second, we adopt an encoder network to learn the relationship between the latent space and the real image space, and use pairwise feature matching to keep the structure of generated images. We experiment with natural images of faces, flowers, and birds, and demonstrate that the proposed models are capable of generating realistic and diverse samples with fine-grained category labels. We further show that our models can be applied to other tasks, such as image inpainting, super-resolution, and data augmentation for training better face recognition models.


page 1

page 7

page 8

page 11

page 12

page 13

page 15


Fine-Grained Image Generation from Bangla Text Description using Attentional Generative Adversarial Network

Generating fine-grained, realistic images from text has many application...

AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks

In this paper, we propose an Attentional Generative Adversarial Network ...

Persuasive Faces: Generating Faces in Advertisements

In this paper, we examine the visual variability of objects across diffe...

Towards Fine-grained Image Classification with Generative Adversarial Networks and Facial Landmark Detection

Fine-grained classification remains a challenging task because distingui...

Latent Space Energy-based Model for Fine-grained Open Set Recognition

Fine-grained open-set recognition (FineOSR) aims to recognize images bel...

Few-shot Knowledge Transfer for Fine-grained Cartoon Face Generation

In this paper, we are interested in generating fine-grained cartoon face...

Reconstructing Faces from fMRI Patterns using Deep Generative Neural Networks

While objects from different categories can be reliably decoded from fMR...

Please sign up or login with your details

Forgot password? Click here to reset