Semantic-aware One-shot Face Re-enactment with Dense Correspondence Estimation

by   Yunfan Liu, et al.

One-shot face re-enactment is a challenging task due to the identity mismatch between source and driving faces. Specifically, the suboptimally disentangled identity information of driving subjects would inevitably interfere with the re-enactment results and lead to face shape distortion. To solve this problem, this paper proposes to use 3D Morphable Model (3DMM) for explicit facial semantic decomposition and identity disentanglement. Instead of using 3D coefficients alone for re-enactment control, we take the advantage of the generative ability of 3DMM to render textured face proxies. These proxies contain abundant yet compact geometric and semantic information of human faces, which enable us to compute the face motion field between source and driving images by estimating the dense correspondence. In this way, we could approximate re-enactment results by warping source images according to the motion field, and a Generative Adversarial Network (GAN) is adopted to further improve the visual quality of warping results. Extensive experiments on various datasets demonstrate the advantages of the proposed method over existing start-of-the-art benchmarks in both identity preservation and re-enactment fulfillment.


page 1

page 2

page 3

page 5

page 6

page 7


Single Source One Shot Reenactment using Weighted motion From Paired Feature Points

Image reenactment is a task where the target object in the source image ...

Face Transformer: Towards High Fidelity and Accurate Face Swapping

Face swapping aims to generate swapped images that fuse the identity of ...

Depth-Aware Generative Adversarial Network for Talking Head Video Generation

Talking head video generation aims to produce a synthetic human face vid...

Liquid Warping GAN with Attention: A Unified Framework for Human Image Synthesis

We tackle human image synthesis, including human motion imitation, appea...

ReliableSwap: Boosting General Face Swapping Via Reliable Supervision

Almost all advanced face swapping approaches use reconstruction as the p...

AutoToon: Automatic Geometric Warping for Face Cartoon Generation

Caricature, a type of exaggerated artistic portrait, amplifies the disti...

Mesh Guided One-shot Face Reenactment using Graph Convolutional Networks

Face reenactment aims to animate a source face image to a different pose...

Please sign up or login with your details

Forgot password? Click here to reset