Self-Supervised Approach for Facial Movement Based Optical Flow

by   Muhannad Alkaddour, et al.

Computing optical flow is a fundamental problem in computer vision. However, deep learning-based optical flow techniques do not perform well for non-rigid movements such as those found in faces, primarily due to lack of the training data representing the fine facial motion. We hypothesize that learning optical flow on face motion data will improve the quality of predicted flow on faces. The aim of this work is threefold: (1) exploring self-supervised techniques to generate optical flow ground truth for face images; (2) computing baseline results on the effects of using face data to train Convolutional Neural Networks (CNN) for predicting optical flow; and (3) using the learned optical flow in micro-expression recognition to demonstrate its effectiveness. We generate optical flow ground truth using facial key-points in the BP4D-Spontaneous dataset. The generated optical flow is used to train the FlowNetS architecture to test its performance on the generated dataset. The performance of FlowNetS trained on face images surpassed that of other optical flow CNN architectures, demonstrating its usefulness. Our optical flow features are further compared with other methods using the STSTNet micro-expression classifier, and the results indicate that the optical flow obtained using this work has promising applications in facial expression analysis.


page 4

page 6

page 8

page 10

page 14


Optical Flow Techniques for Facial Expression Analysis: Performance Evaluation and Improvements

Optical flow techniques are becoming increasingly performant and robust ...

Unsupervised Generation of Optical Flow Datasets from Videos in the Wild

Dense optical flow ground truth of non-rigid motion for real-world image...

Automatic Quantification of Facial Asymmetry using Facial Landmarks

One-sided facial paralysis causes uneven movements of facial muscles on ...

Face Animation with Multiple Source Images

Face animation has received a lot of attention from researchers in recen...

Versatile Face Animator: Driving Arbitrary 3D Facial Avatar in RGBD Space

Creating realistic 3D facial animation is crucial for various applicatio...

Mesh Guided One-shot Face Reenactment using Graph Convolutional Networks

Face reenactment aims to animate a source face image to a different pose...

Distinguishing Posed and Spontaneous Smiles by Facial Dynamics

Smile is one of the key elements in identifying emotions and present sta...

Please sign up or login with your details

Forgot password? Click here to reset