Towards NIR-VIS Masked Face Recognition

by   Hang Du, et al.

Near-infrared to visible (NIR-VIS) face recognition is the most common case in heterogeneous face recognition, which aims to match a pair of face images captured from two different modalities. Existing deep learning based methods have made remarkable progress in NIR-VIS face recognition, while it encounters certain newly-emerged difficulties during the pandemic of COVID-19, since people are supposed to wear facial masks to cut off the spread of the virus. We define this task as NIR-VIS masked face recognition, and find it problematic with the masked face in the NIR probe image. First, the lack of masked face data is a challenging issue for the network training. Second, most of the facial parts (cheeks, mouth, nose etc.) are fully occluded by the mask, which leads to a large amount of loss of information. Third, the domain gap still exists in the remaining facial parts. In such scenario, the existing methods suffer from significant performance degradation caused by the above issues. In this paper, we aim to address the challenge of NIR-VIS masked face recognition from the perspectives of training data and training method. Specifically, we propose a novel heterogeneous training method to maximize the mutual information shared by the face representation of two domains with the help of semi-siamese networks. In addition, a 3D face reconstruction based approach is employed to synthesize masked face from the existing NIR image. Resorting to these practices, our solution provides the domain-invariant face representation which is also robust to the mask occlusion. Extensive experiments on three NIR-VIS face datasets demonstrate the effectiveness and cross-dataset-generalization capacity of our method.


page 1

page 3


When Face Recognition Meets Occlusion: A New Benchmark

The existing face recognition datasets usually lack occlusion samples, w...

Multi-Dataset Benchmarks for Masked Identification using Contrastive Representation Learning

The COVID-19 pandemic has drastically changed accepted norms globally. W...

A study of Biases in Common Face Recognition Datasets

The performance of commonly used facial recognition datasets have been s...

Semi-Siamese Training for Shallow Face Learning

Most existing public face datasets, such as MS-Celeb-1M and VGGFace2, pr...

Multi-Agent Semi-Siamese Training for Long-tail and Shallow Face Learning

With the recent development of deep convolutional neural networks and la...

Facial Feature Embedded CycleGAN for VIS-NIR Translation

VIS-NIR face recognition remains a challenging task due to the distincti...

Sparse Graphical Representation based Discriminant Analysis for Heterogeneous Face Recognition

Face images captured in heterogeneous environments, e.g., sketches gener...

Please sign up or login with your details

Forgot password? Click here to reset