CharacterGAN: Few-Shot Keypoint Character Animation and Reposing

02/05/2021
by   Tobias Hinz, et al.
0

We introduce CharacterGAN, a generative model that can be trained on only a few samples (8 - 15) of a given character. Our model generates novel poses based on keypoint locations, which can be modified in real time while providing interactive feedback, allowing for intuitive reposing and animation. Since we only have very limited training samples, one of the key challenges lies in how to address (dis)occlusions, e.g. when a hand moves behind or in front of a body. To address this, we introduce a novel layering approach which explicitly splits the input keypoints into different layers which are processed independently. These layers represent different parts of the character and provide a strong implicit bias that helps to obtain realistic results even with strong (dis)occlusions. To combine the features of individual layers we use an adaptive scaling approach conditioned on all keypoints. Finally, we introduce a mask connectivity constraint to reduce distortion artifacts that occur with extreme out-of-distribution poses at test time. We show that our approach outperforms recent baselines and creates realistic animations for diverse characters. We also show that our model can handle discrete state changes, for example a profile facing left or right, that the different layers do indeed learn features specific for the respective keypoints in those layers, and that our model scales to larger datasets when more data is available.

READ FULL TEXT

page 3

page 6

page 7

research
03/29/2017

Click Here: Human-Localized Keypoints as Guidance for Viewpoint Estimation

We motivate and address a human-in-the-loop variant of the monocular vie...
research
01/06/2012

Interactive Character Posing by Sparse Coding

Character posing is of interest in computer animation. It is difficult d...
research
04/01/2019

Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters

We introduce a novel approach for keypoint detection task that combines ...
research
05/25/2023

Zero-shot Generation of Training Data with Denoising Diffusion Probabilistic Model for Handwritten Chinese Character Recognition

There are more than 80,000 character categories in Chinese while most of...
research
10/04/2019

Neural Puppet: Generative Layered Cartoon Characters

We propose a learning based method for generating new animations of a ca...
research
06/04/2022

APES: Articulated Part Extraction from Sprite Sheets

Rigged puppets are one of the most prevalent representations to create 2...
research
03/25/2023

PAniC-3D: Stylized Single-view 3D Reconstruction from Portraits of Anime Characters

We propose PAniC-3D, a system to reconstruct stylized 3D character heads...

Please sign up or login with your details

Forgot password? Click here to reset