Cycle Encoding of a StyleGAN Encoder for Improved Reconstruction and Editability

by   Xudong Mao, et al.

GAN inversion aims to invert an input image into the latent space of a pre-trained GAN. Despite the recent advances in GAN inversion, there remain challenges to mitigate the tradeoff between distortion and editability, i.e. reconstructing the input image accurately and editing the inverted image with a small visual quality drop. The recently proposed pivotal tuning model makes significant progress towards reconstruction and editability, by using a two-step approach that first inverts the input image into a latent code, called pivot code, and then alters the generator so that the input image can be accurately mapped into the pivot code. Here, we show that both reconstruction and editability can be improved by a proper design of the pivot code. We present a simple yet effective method, named cycle encoding, for a high-quality pivot code. The key idea of our method is to progressively train an encoder in varying spaces according to a cycle scheme: W->W+->W. This training methodology preserves the properties of both W and W+ spaces, i.e. high editability of W and low distortion of W+. To further decrease the distortion, we also propose to refine the pivot code with an optimization-based method, where a regularization term is introduced to reduce the degradation in editability. Qualitative and quantitative comparisons to several state-of-the-art methods demonstrate the superiority of our approach.


page 12

page 14

page 15

page 17

page 18

page 21

page 22

page 24


Designing an Encoder for StyleGAN Image Manipulation

Recently, there has been a surge of diverse methods for performing image...

HyperInverter: Improving StyleGAN Inversion via Hypernetwork

Real-world image manipulation has achieved fantastic progress in recent ...

Make Encoder Great Again in 3D GAN Inversion through Geometry and Occlusion-Aware Encoding

3D GAN inversion aims to achieve high reconstruction fidelity and reason...

3D-Aware Encoding for Style-based Neural Radiance Fields

We tackle the task of NeRF inversion for style-based neural radiance fie...

Neural Image Compression Using Masked Sparse Visual Representation

We study neural image compression based on the Sparse Visual Representat...

IntereStyle: Encoding an Interest Region for Robust StyleGAN Inversion

Recently, manipulation of real-world images has been highly elaborated a...

DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs

Close-up facial images captured at close distances often suffer from per...

Please sign up or login with your details

Forgot password? Click here to reset