COCO-GAN: Generation by Parts via Conditional Coordinating

by   Chieh Hubert Lin, et al.
National Tsing Hua University

Humans can only interact with part of the surrounding environment due to biological restrictions. Therefore, we learn to reason the spatial relationships across a series of observations to piece together the surrounding environment. Inspired by such behavior and the fact that machines also have computational constraints, we propose COnditional COordinate GAN (COCO-GAN) of which the generator generates images by parts based on their spatial coordinates as the condition. On the other hand, the discriminator learns to justify realism across multiple assembled patches by global coherence, local appearance, and edge-crossing continuity. Despite the full images are never generated during training, we show that COCO-GAN can produce state-of-the-art-quality full images during inference. We further demonstrate a variety of novel applications enabled by teaching the network to be aware of coordinates. First, we perform extrapolation to the learned coordinate manifold and generate off-the-boundary patches. Combining with the originally generated full image, COCO-GAN can produce images that are larger than training samples, which we called "beyond-boundary generation". We then showcase panorama generation within a cylindrical coordinate system that inherently preserves horizontally cyclic topology. On the computation side, COCO-GAN has a built-in divide-and-conquer paradigm that reduces memory requisition during training and inference, provides high-parallelism, and can generate parts of images on-demand.


page 13

page 14

page 15

page 16

page 17

page 18

page 20

page 22


Reducing Training Sample Memorization in GANs by Training with Memorization Rejection

Generative adversarial network (GAN) continues to be a popular research ...

Guiding GANs: How to control non-conditional pre-trained GANs for conditional image generation

Generative Adversarial Networks (GANs) are an arrange of two neural netw...

InsetGAN for Full-Body Image Generation

While GANs can produce photo-realistic images in ideal conditions for ce...

Feasibility of Inconspicuous GAN-generated Adversarial Patches against Object Detection

Standard approaches for adversarial patch generation lead to noisy consp...

Instance-Conditioned GAN

Generative Adversarial Networks (GANs) can generate near photo realistic...

Unsupervised Holistic Image Generation from Key Local Patches

We introduce a new problem of generating an image based on a small numbe...

Please sign up or login with your details

Forgot password? Click here to reset