Towards transformation-resilient provenance detection of digital media

by   Jamie Hayes, et al.

Advancements in deep generative models have made it possible to synthesize images, videos and audio signals that are difficult to distinguish from natural signals, creating opportunities for potential abuse of these capabilities. This motivates the problem of tracking the provenance of signals, i.e., being able to determine the original source of a signal. Watermarking the signal at the time of signal creation is a potential solution, but current techniques are brittle and watermark detection mechanisms can easily be bypassed by applying post-processing transformations (cropping images, shifting pitch in the audio etc.). In this paper, we introduce ReSWAT (Resilient Signal Watermarking via Adversarial Training), a framework for learning transformation-resilient watermark detectors that are able to detect a watermark even after a signal has been through several post-processing transformations. Our detection method can be applied to domains with continuous data representations such as images, videos or sound signals. Experiments on watermarking image and audio signals show that our method can reliably detect the provenance of a signal, even if it has been through several post-processing transformations, and improve upon related work in this setting. Furthermore, we show that for specific kinds of transformations (perturbations bounded in the L2 norm), we can even get formal guarantees on the ability of our model to detect the watermark. We provide qualitative examples of watermarked image and audio samples in


page 12

page 16

page 19


Audio Defect Detection in Music with Deep Networks

With increasing amounts of music being digitally transferred from produc...

Evaluation of post-processing algorithms for polyphonic sound event detection

Sound event detection (SED) aims at identifying audio events (audio tagg...

Effects of Pre- and Post-Processing on type-based Embeddings in Lexical Semantic Change Detection

Lexical semantic change detection is a new and innovative research field...

JWST MIRI Imaging Data Post-Processing Preliminary Study with Fourier Transformation to uncover potentially celestial-origin signals

This manuscript reports a part of a dedicated study aiming to disentangl...

WaveFake: A Data Set to Facilitate Audio Deepfake Detection

Deep generative modeling has the potential to cause significant harm to ...

Improving Post-Processing of Audio Event Detectors Using Reinforcement Learning

We apply post-processing to the class probability distribution outputs o...

Effects of Word-frequency based Pre- and Post- Processings for Audio Captioning

The system we used for Task 6 (Automated Audio Captioning)of the Detecti...

Please sign up or login with your details

Forgot password? Click here to reset