Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition

by   Kai Wang, et al.

Occlusion and pose variations, which can change facial appearance significantly, are two major obstacles for automatic Facial Expression Recognition (FER). Though automatic FER has made substantial progresses in the past few decades, occlusion-robust and pose-invariant issues of FER have received relatively less attention, especially in real-world scenarios. This paper addresses the real-world pose and occlusion robust FER problem with three-fold contributions. First, to stimulate the research of FER under real-world occlusions and variant poses, we build several in-the-wild facial expression datasets with manual annotations for the community. Second, we propose a novel Region Attention Network (RAN), to adaptively capture the importance of facial regions for occlusion and pose variant FER. The RAN aggregates and embeds varied number of region features produced by a backbone convolutional neural network into a compact fixed-length representation. Last, inspired by the fact that facial expressions are mainly defined by facial action units, we propose a region biased loss to encourage high attention weights for the most important regions. We validate our RAN and region biased loss on both our built test datasets and four popular datasets: FERPlus, AffectNet, RAF-DB, and SFEW. Extensive experiments show that our RAN and region biased loss largely improve the performance of FER with occlusion and variant pose. Our method also achieves state-of-the-art results on FERPlus, AffectNet, RAF-DB, and SFEW. Code and the collected test data will be publicly available.


page 1

page 5

page 7

page 8


Region attention and graph embedding network for occlusion objective class-based micro-expression recognition

Micro-expression recognition (MER) has attracted lots of researchers' at...

Robust Facial Expression Recognition with Convolutional Visual Transformers

Facial Expression Recognition (FER) in the wild is extremely challenging...

Facial Expression Analysis under Partial Occlusion: A Survey

Automatic machine-based Facial Expression Analysis (FEA) has made substa...

AU-Aware Vision Transformers for Biased Facial Expression Recognition

Studies have proven that domain bias and label bias exist in different F...

Point Adversarial Self Mining: A Simple Method for Facial Expression Recognition in the Wild

In this paper, the Point Adversarial Self Mining (PASM) approach, a simp...

Pose-adaptive Hierarchical Attention Network for Facial Expression Recognition

Multi-view facial expression recognition (FER) is a challenging task bec...

Landmark Guidance Independent Spatio-channel Attention and Complementary Context Information based Facial Expression Recognition

A recent trend to recognize facial expressions in the real-world scenari...

Please sign up or login with your details

Forgot password? Click here to reset