WOUAF: Weight Modulation for User Attribution and Fingerprinting in Text-to-Image Diffusion Models

by   Changhoon Kim, et al.

The rapid advancement of generative models, facilitating the creation of hyper-realistic images from textual descriptions, has concurrently escalated critical societal concerns such as misinformation. Traditional fake detection mechanisms, although providing some mitigation, fall short in attributing responsibility for the malicious use of synthetic images. This paper introduces a novel approach to model fingerprinting that assigns responsibility for the generated images, thereby serving as a potential countermeasure to model misuse. Our method modifies generative models based on each user's unique digital fingerprint, imprinting a unique identifier onto the resultant content that can be traced back to the user. This approach, incorporating fine-tuning into Text-to-Image (T2I) tasks using the Stable Diffusion Model, demonstrates near-perfect attribution accuracy with a minimal impact on output quality. We rigorously scrutinize our method's secrecy under two distinct scenarios: one where a malicious user attempts to detect the fingerprint, and another where a user possesses a comprehensive understanding of our method. We also evaluate the robustness of our approach against various image post-processing manipulations typically executed by end-users. Through extensive evaluation of the Stable Diffusion models, our method presents a promising and novel avenue for accountable model distribution and responsible use.


page 7

page 14

page 15

page 17

page 18

page 19

page 20

page 21


Responsible Disclosure of Generative Models Using Scalable Fingerprinting

Over the past six years, deep generative models have achieved a qualitat...

Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images

State-of-the-art (SOTA) Generative Models (GMs) can synthesize photo-rea...

The Stable Signature: Rooting Watermarks in Latent Diffusion Models

Generative image modeling enables a wide range of applications but raise...

Inspecting the Geographical Representativeness of Images from Text-to-Image Models

Recent progress in generative models has resulted in models that produce...

Attributable-Watermarking of Speech Generative Models

Generative models are now capable of synthesizing images, speeches, and ...

Attributing Image Generative Models using Latent Fingerprints

Generative models have enabled the creation of contents that are indisti...

Manipulating Embeddings of Stable Diffusion Prompts

Generative text-to-image models such as Stable Diffusion allow users to ...

Please sign up or login with your details

Forgot password? Click here to reset