PetsGAN: Rethinking Priors for Single Image Generation

by   Zicheng Zhang, et al.

Single image generation (SIG), described as generating diverse samples that have similar visual content with the given single image, is first introduced by SinGAN which builds a pyramid of GANs to progressively learn the internal patch distribution of the single image. It also shows great potentials in a wide range of image manipulation tasks. However, the paradigm of SinGAN has limitations in terms of generation quality and training time. Firstly, due to the lack of high-level information, SinGAN cannot handle the object images well as it does on the scene and texture images. Secondly, the separate progressive training scheme is time-consuming and easy to cause artifact accumulation. To tackle these problems, in this paper, we dig into the SIG problem and improve SinGAN by fully-utilization of internal and external priors. The main contributions of this paper include: 1) We introduce to SIG a regularized latent variable model. To the best of our knowledge, it is the first time to give a clear formulation and optimization goal of SIG, and all the existing methods for SIG can be regarded as special cases of this model. 2) We design a novel Prior-based end-to-end training GAN (PetsGAN) to overcome the problems of SinGAN. Our method gets rid of the time-consuming progressive training scheme and can be trained end-to-end. 3) We construct abundant qualitative and quantitative experiments to show the superiority of our method on both generated image quality, diversity, and the training speed. Moreover, we apply our method to other image manipulation tasks (e.g., style transfer, harmonization), and the results further prove the effectiveness and efficiency of our method.


page 1

page 2

page 6

page 7


SinGAN: Learning a Generative Model from a Single Natural Image

We introduce SinGAN, an unconditional generative model that can be learn...

SinDiffusion: Learning a Diffusion Model from a Single Natural Image

We present SinDiffusion, leveraging denoising diffusion models to captur...

SinDDM: A Single Image Denoising Diffusion Model

Denoising diffusion models (DDMs) have led to staggering performance lea...

Meta Internal Learning

Internal learning for single-image generation is a framework, where a ge...

Drop the GAN: In Defense of Patches Nearest Neighbors as Single Image Generative Models

Single image generative models perform synthesis and manipulation tasks ...

DEff-GAN: Diverse Attribute Transfer for Few-Shot Image Synthesis

Requirements of large amounts of data is a difficulty in training many G...

Semantic Pyramid for Image Generation

We present a novel GAN-based model that utilizes the space of deep featu...

Please sign up or login with your details

Forgot password? Click here to reset