GLEAN: Generative Latent Bank for Image Super-Resolution and Beyond

by   Kelvin C. K. Chan, et al.
Nanyang Technological University

We show that pre-trained Generative Adversarial Networks (GANs) such as StyleGAN and BigGAN can be used as a latent bank to improve the performance of image super-resolution. While most existing perceptual-oriented approaches attempt to generate realistic outputs through learning with adversarial loss, our method, Generative LatEnt bANk (GLEAN), goes beyond existing practices by directly leveraging rich and diverse priors encapsulated in a pre-trained GAN. But unlike prevalent GAN inversion methods that require expensive image-specific optimization at runtime, our approach only needs a single forward pass for restoration. GLEAN can be easily incorporated in a simple encoder-bank-decoder architecture with multi-resolution skip connections. Employing priors from different generative models allows GLEAN to be applied to diverse categories ( human faces, cats, buildings, and cars). We further present a lightweight version of GLEAN, named LightGLEAN, which retains only the critical components in GLEAN. Notably, LightGLEAN consists of only 21 parameters and 35 our method to different tasks including image colorization and blind image restoration, and extensive experiments show that our proposed models perform favorably in comparison to existing methods. Codes and models are available at


page 6

page 8

page 9

page 11

page 15

page 16

page 17

page 18


GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution

We show that pre-trained Generative Adversarial Networks (GANs), e.g., S...

Physics-Based Generative Adversarial Models for Image Restoration and Beyond

We present an algorithm to directly solve numerous image restoration pro...

Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution

Single image super-resolution (SISR) with generative adversarial network...

Latent Multi-Relation Reasoning for GAN-Prior based Image Super-Resolution

Recently, single image super-resolution (SR) under large scaling factors...

Bayesian Image Reconstruction using Deep Generative Models

Machine learning models are commonly trained end-to-end and in a supervi...

Image-Adaptive GAN based Reconstruction

In the recent years, there has been a significant improvement in the qua...

Unlocking Masked Autoencoders as Loss Function for Image and Video Restoration

Image and video restoration has achieved a remarkable leap with the adve...

Please sign up or login with your details

Forgot password? Click here to reset