Instance-Conditioned GAN

by   Arantxa Casanova, et al.

Generative Adversarial Networks (GANs) can generate near photo realistic images in narrow domains such as human faces. Yet, modeling complex distributions of datasets such as ImageNet and COCO-Stuff remains challenging in unconditional settings. In this paper, we take inspiration from kernel density estimation techniques and introduce a non-parametric approach to modeling distributions of complex datasets. We partition the data manifold into a mixture of overlapping neighborhoods described by a datapoint and its nearest neighbors, and introduce a model, called instance-conditioned GAN (IC-GAN), which learns the distribution around each datapoint. Experimental results on ImageNet and COCO-Stuff show that IC-GAN significantly improves over unconditional models and unsupervised data partitioning baselines. Moreover, we show that IC-GAN can effortlessly transfer to datasets not seen during training by simply changing the conditioning instances, and still generate realistic images. Finally, we extend IC-GAN to the class-conditional case and show semantically controllable generation and competitive quantitative results on ImageNet; while improving over BigGAN on ImageNet-LT. We will opensource our code and trained models to reproduce the reported results.


page 2

page 19

page 20

page 22

page 23

page 25

page 26

page 29


Spider GAN: Leveraging Friendly Neighbors to Accelerate GAN Training

Training Generative adversarial networks (GANs) stably is a challenging ...

Unsupervised Controllable Generation with Self-Training

Recent generative adversarial networks (GANs) are able to generate impre...

Generative Adversarial Forests for Better Conditioned Adversarial Learning

In recent times, many of the breakthroughs in various vision-related tas...

On the Effects of Batch and Weight Normalization in Generative Adversarial Networks

Generative adversarial networks (GANs) are highly effective unsupervised...

Class-Distinct and Class-Mutual Image Generation with GANs

We describe a new problem called class-distinct and class-mutual (DM) im...

COCO-GAN: Generation by Parts via Conditional Coordinating

Humans can only interact with part of the surrounding environment due to...

Instance Selection for GANs

Recent advances in Generative Adversarial Networks (GANs) have led to th...

Please sign up or login with your details

Forgot password? Click here to reset