CapsField: Light Field-based Face and Expression Recognition in the Wild using Capsule Routing

by   Alireza Sepas-Moghaddam, et al.

Light field (LF) cameras provide rich spatio-angular visual representations by sensing the visual scene from multiple perspectives and have recently emerged as a promising technology to boost the performance of human-machine systems such as biometrics and affective computing. Despite the significant success of LF representation for constrained facial image analysis, this technology has never been used for face and expression recognition in the wild. In this context, this paper proposes a new deep face and expression recognition solution, called CapsField, based on a convolutional neural network and an additional capsule network that utilizes dynamic routing to learn hierarchical relations between capsules. CapsField extracts the spatial features from facial images and learns the angular part-whole relations for a selected set of 2D sub-aperture images rendered from each LF image. To analyze the performance of the proposed solution in the wild, the first in the wild LF face dataset, along with a new complementary constrained face dataset captured from the same subjects recorded earlier have been captured and are made available. A subset of the in the wild dataset contains facial images with different expressions, annotated for usage in the context of face expression recognition tests. An extensive performance assessment study using the new datasets has been conducted for the proposed and relevant prior solutions, showing that the CapsField proposed solution achieves superior performance for both face and expression recognition tasks when compared to the state-of-the-art.


page 1

page 3

page 6

page 8

page 9

page 16


A Double-Deep Spatio-Angular Learning Framework for Light Field based Face Recognition

Face recognition has attracted increasing attention due to its wide rang...

Real-time Facial Expression Recognition "In The Wild” by Disentangling 3D Expression from Identity

Human emotions analysis has been the focus of many studies, especially i...

E2-Capsule Neural Networks for Facial Expression Recognition Using AU-Aware Attention

Capsule neural network is a new and popular technique in deep learning. ...

Novel Long Short-Term Memory Cell Architectures: Application to Light Field Face Recognition

With the emergence of lenslet light field cameras able to capture rich s...

Multi-Perspective LSTM for Joint Visual Representation Learning

We present a novel LSTM cell architecture capable of learning both intra...

Affect Expression Behaviour Analysis in the Wild using Spatio-Channel Attention and Complementary Context Information

Facial expression recognition(FER) in the wild is crucial for building r...

Expression Recognition Using the Periocular Region: A Feasibility Study

This paper investigates the feasibility of using the periocular region f...

Please sign up or login with your details

Forgot password? Click here to reset