Transformer-Based Speech Synthesizer Attribution in an Open Set Scenario

by   Emily R. Bartusiak, et al.

Speech synthesis methods can create realistic-sounding speech, which may be used for fraud, spoofing, and misinformation campaigns. Forensic methods that detect synthesized speech are important for protection against such attacks. Forensic attribution methods provide even more information about the nature of synthesized speech signals because they identify the specific speech synthesis method (i.e., speech synthesizer) used to create a speech signal. Due to the increasing number of realistic-sounding speech synthesizers, we propose a speech attribution method that generalizes to new synthesizers not seen during training. To do so, we investigate speech synthesizer attribution in both a closed set scenario and an open set scenario. In other words, we consider some speech synthesizers to be "known" synthesizers (i.e., part of the closed set) and others to be "unknown" synthesizers (i.e., part of the open set). We represent speech signals as spectrograms and train our proposed method, known as compact attribution transformer (CAT), on the closed set for multi-class classification. Then, we extend our analysis to the open set to attribute synthesized speech signals to both known and unknown synthesizers. We utilize a t-distributed stochastic neighbor embedding (tSNE) on the latent space of the trained CAT to differentiate between each unknown synthesizer. Additionally, we explore poly-1 loss formulations to improve attribution results. Our proposed approach successfully attributes synthesized speech signals to their respective speech synthesizers in both closed and open set scenarios.


Synthesized Speech Detection Using Convolutional Transformer-Based Spectrogram Analysis

Synthesized speech is common today due to the prevalence of virtual assi...

Open Set Synthetic Image Source Attribution

AI-generated images have become increasingly realistic and have garnered...

Open Set Authorship Attribution toward Demystifying Victorian Periodicals

Existing research in computational authorship attribution (AA) has prima...

Progressive Open Space Expansion for Open-Set Model Attribution

Despite the remarkable progress in generative technology, the Janus-face...

Syn-Att: Synthetic Speech Attribution via Semi-Supervised Unknown Multi-Class Ensemble of CNNs

With the huge technological advances introduced by deep learning in audi...

Attributable-Watermarking of Speech Generative Models

Generative models are now capable of synthesizing images, speeches, and ...

Are Social Networks Watermarking Us or Are We (Unawarely) Watermarking Ourself?

In the last decade, Social Networks (SNs) have deeply changed many aspec...

Please sign up or login with your details

Forgot password? Click here to reset