Self-Learning Transformations for Improving Gaze and Head Redirection

by   Yufeng Zheng, et al.

Many computer vision tasks rely on labeled data. Rapid progress in generative modeling has led to the ability to synthesize photorealistic images. However, controlling specific aspects of the generation process such that the data can be used for supervision of downstream tasks remains challenging. In this paper we propose a novel generative model for images of faces, that is capable of producing high-quality images under fine-grained control over eye gaze and head orientation angles. This requires the disentangling of many appearance related factors including gaze and head orientation but also lighting, hue etc. We propose a novel architecture which learns to discover, disentangle and encode these extraneous variations in a self-learned manner. We further show that explicitly disentangling task-irrelevant factors results in more accurate modelling of gaze and head orientation. A novel evaluation scheme shows that our method improves upon the state-of-the-art in redirection accuracy and disentanglement between gaze direction and head orientation changes. Furthermore, we show that in the presence of limited amounts of real-world training data, our method allows for improvements in the downstream task of semi-supervised cross-dataset gaze estimation. Please check our project page at:


page 8

page 11

page 17

page 18


NeRF-Gaze: A Head-Eye Redirection Parametric Model for Gaze Estimation

Gaze estimation is the fundamental basis for many visual tasks. Yet, the...

ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation

Gaze estimation is a fundamental task in many applications of computer v...

EyeBAG: Accurate Control of Eye Blink and Gaze Based on Data Augmentation Leveraging Style Mixing

Recent developments in generative models have enabled the generation of ...

Prioritized Kinematic Control of Joint-Constrained Head-Eye Robots using the Intermediate Value Approach

Existing gaze controllers for head-eye robots can only handle single fix...

MTGLS: Multi-Task Gaze Estimation with Limited Supervision

Robust gaze estimation is a challenging task, even for deep CNNs, due to...

Toward Super-Resolution for Appearance-Based Gaze Estimation

Gaze tracking is a valuable tool with a broad range of applications in v...

Angle Range and Identity Similarity Enhanced Gaze and Head Redirection based on Synthetic data

In this paper, we propose a method for improving the angular accuracy an...

Please sign up or login with your details

Forgot password? Click here to reset