DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling

by   Xiaoguang Han, et al.

Face modeling has been paid much attention in the field of visual computing. There exist many scenarios, including cartoon characters, avatars for social media, 3D face caricatures as well as face-related art and design, where low-cost interactive face modeling is a popular approach especially among amateur users. In this paper, we propose a deep learning based sketching system for 3D face and caricature modeling. This system has a labor-efficient sketching interface, that allows the user to draw freehand imprecise yet expressive 2D lines representing the contours of facial features. A novel CNN based deep regression network is designed for inferring 3D face models from 2D sketches. Our network fuses both CNN and shape based features of the input sketch, and has two independent branches of fully connected layers generating independent subsets of coefficients for a bilinear face representation. Our system also supports gesture based interactions for users to further manipulate initial face models. Both user studies and numerical results indicate that our sketching system can help users create face models quickly and effectively. A significantly expanded face database with diverse identities, expressions and levels of exaggeration is constructed to promote further research and evaluation of face modeling techniques.


page 5

page 7

page 9

page 10


Deep learning for identification and face, gender, expression recognition under constraints

Biometric recognition based on the full face is an extensive research ar...

Shape-from-Mask: A Deep Learning Based Human Body Shape Reconstruction from Binary Mask Images

3D content creation is referred to as one of the most fundamental tasks ...

Learning Robust Deep Face Representation

With the development of convolution neural network, more and more resear...

Privacy Partitioning: Protecting User Data During the Deep Learning Inference Phase

We present a practical method for protecting data during the inference p...

Mobile Face Tracking: A Survey and Benchmark

With the rapid development of smartphones, facial analysis has been play...

Robustness Analysis of Face Obscuration

Face obscuration is often needed by law enforcement or mass media outlet...

JNR: Joint-based Neural Rig Representation for Compact 3D Face Modeling

In this paper, we introduce a novel approach to learn a 3D face model us...

Please sign up or login with your details

Forgot password? Click here to reset