Adaptively Lighting up Facial Expression Crucial Regions via Local Non-Local Joint Network

by   Shasha Mao, et al.

Facial expression recognition (FER) is still one challenging research due to the small inter-class discrepancy in the facial expression data. In view of the significance of facial crucial regions for FER, many existing researches utilize the prior information from some annotated crucial points to improve the performance of FER. However, it is complicated and time-consuming to manually annotate facial crucial points, especially for vast wild expression images. Based on this, a local non-local joint network is proposed to adaptively light up the facial crucial regions in feature learning of FER in this paper. In the proposed method, two parts are constructed based on facial local and non-local information respectively, where an ensemble of multiple local networks are proposed to extract local features corresponding to multiple facial local regions and a non-local attention network is addressed to explore the significance of each local region. Especially, the attention weights obtained by the non-local network is fed into the local part to achieve the interactive feedback between the facial global and local information. Interestingly, the non-local weights corresponding to local regions are gradually updated and higher weights are given to more crucial regions. Moreover, U-Net is employed to extract the integrated features of deep semantic information and low hierarchical detail information of expression images. Finally, experimental results illustrate that the proposed method achieves more competitive performance compared with several state-of-the art methods on five benchmark datasets. Noticeably, the analyses of the non-local weights corresponding to local regions demonstrate that the proposed method can automatically enhance some crucial regions in the process of feature learning without any facial landmark information.


page 1

page 2

page 9

page 10

page 13

page 15


Facial Motion Prior Networks for Facial Expression Recognition

Deep learning based facial expression recognition (FER) has received a l...

Kinship Verification Based on Cross-Generation Feature Interaction Learning

Kinship verification from facial images has been recognized as an emergi...

Machine vision detection to daily facial fatigue with a nonlocal 3D attention network

Fatigue detection is valued for people to keep mental health and prevent...

Interpretable Facial Relational Network Using Relational Importance

Human face analysis is an important task in computer vision. According t...

Affect Expression Behaviour Analysis in the Wild using Spatio-Channel Attention and Complementary Context Information

Facial expression recognition(FER) in the wild is crucial for building r...

Hierarchical Prior Mining for Non-local Multi-View Stereo

As a fundamental problem in computer vision, multi-view stereo (MVS) aim...

Point Adversarial Self Mining: A Simple Method for Facial Expression Recognition in the Wild

In this paper, the Point Adversarial Self Mining (PASM) approach, a simp...

Please sign up or login with your details

Forgot password? Click here to reset