End-to-end Face-swapping via Adaptive Latent Representation Learning

by   Chenhao Lin, et al.
Xi'an Jiaotong University

Taking full advantage of the excellent performance of StyleGAN, style transfer-based face swapping methods have been extensively investigated recently. However, these studies require separate face segmentation and blending modules for successful face swapping, and the fixed selection of the manipulated latent code in these works is reckless, thus degrading face swapping quality, generalizability, and practicability. This paper proposes a novel and end-to-end integrated framework for high resolution and attribute preservation face swapping via Adaptive Latent Representation Learning. Specifically, we first design a multi-task dual-space face encoder by sharing the underlying feature extraction network to simultaneously complete the facial region perception and face encoding. This encoder enables us to control the face pose and attribute individually, thus enhancing the face swapping quality. Next, we propose an adaptive latent codes swapping module to adaptively learn the mapping between the facial attributes and the latent codes and select effective latent codes for improved retention of facial attributes. Finally, the initial face swapping image generated by StyleGAN2 is blended with the facial region mask generated by our encoder to address the background blur problem. Our framework integrating facial perceiving and blending into the end-to-end training and testing process can achieve high realistic face-swapping on wild faces without segmentation masks. Experimental results demonstrate the superior performance of our approach over state-of-the-art methods.


page 1

page 3

page 4

page 5

page 7

page 8


Unconstrained Face Sketch Synthesis via Perception-Adaptive Network and A New Benchmark

Face sketch generation has attracted much attention in the field of visu...

FlowFace++: Explicit Semantic Flow-supervised End-to-End Face Swapping

This work proposes a novel face-swapping framework FlowFace++, utilizing...

A Deep Face Identification Network Enhanced by Facial Attributes Prediction

In this paper, we propose a new deep framework which predicts facial att...

Mask-aware Photorealistic Face Attribute Manipulation

The task of face attribute manipulation has found increasing application...

LR-to-HR Face Hallucination with an Adversarial Progressive Attribute-Induced Network

Face super-resolution is a challenging and highly ill-posed problem sinc...

An End-to-end Method for Producing Scanning-robust Stylized QR Codes

Quick Response (QR) code is one of the most worldwide used two-dimension...

Reinforced Disentanglement for Face Swapping without Skip Connection

The SOTA face swap models still suffer the problem of either target iden...

Please sign up or login with your details

Forgot password? Click here to reset