Exposing Deepfake with Pixel-wise AR and PPG Correlation from Faint Signals

by   Maoyu Mao, et al.

Deepfake poses a serious threat to the reliability of judicial evidence and intellectual property protection. In spite of an urgent need for Deepfake identification, existing pixel-level detection methods are increasingly unable to resist the growing realism of fake videos and lack generalization. In this paper, we propose a scheme to expose Deepfake through faint signals hidden in face videos. This scheme extracts two types of minute information hidden between face pixels-photoplethysmography (PPG) features and auto-regressive (AR) features, which are used as the basis for forensics in the temporal and spatial domains, respectively. According to the principle of PPG, tracking the absorption of light by blood cells allows remote estimation of the temporal domains heart rate (HR) of face video, and irregular HR fluctuations can be seen as traces of tampering. On the other hand, AR coefficients are able to reflect the inter-pixel correlation, and can also reflect the traces of smoothing caused by up-sampling in the process of generating fake faces. Furthermore, the scheme combines asymmetric convolution block (ACBlock)-based improved densely connected networks (DenseNets) to achieve face video authenticity forensics. Its asymmetric convolutional structure enhances the robustness of network to the input feature image upside-down and left-right flipping, so that the sequence of feature stitching does not affect detection results. Simulation results show that our proposed scheme provides more accurate authenticity detection results on multiple deep forgery datasets and has better generalization compared to the benchmark strategy.


page 1

page 2

page 3

page 4

page 6

page 7

page 8

page 9


VideoFACT: Detecting Video Forgeries Using Attention, Scene Context, and Forensic Traces

Fake videos represent an important misinformation threat. While existing...

An Audio-Visual Attention Based Multimodal Network for Fake Talking Face Videos Detection

DeepFake based digital facial forgery is threatening the public media se...

Towards Accurate and Reliable Change Detection of Remote Sensing Images via Knowledge Review and Online Uncertainty Estimation

Change detection (CD) is an essential task for various real-world applic...

Watch Out for the Confusing Faces: Detecting Face Swapping with the Probability Distribution of Face Identification Models

Recently, face swapping has been developing rapidly and achieved a surpr...

Where Do Deep Fakes Look? Synthetic Face Detection via Gaze Tracking

Following the recent initiatives for the democratization of AI, deep fak...

Towards Interpretable and Robust Hand Detection via Pixel-wise Prediction

The lack of interpretability of existing CNN-based hand detection method...

Please sign up or login with your details

Forgot password? Click here to reset