Dataset Inference for Self-Supervised Models

by   Adam Dziedzic, et al.

Self-supervised models are increasingly prevalent in machine learning (ML) since they reduce the need for expensively labeled data. Because of their versatility in downstream applications, they are increasingly used as a service exposed via public APIs. At the same time, these encoder models are particularly vulnerable to model stealing attacks due to the high dimensionality of vector representations they output. Yet, encoders remain undefended: existing mitigation strategies for stealing attacks focus on supervised learning. We introduce a new dataset inference defense, which uses the private training set of the victim encoder model to attribute its ownership in the event of stealing. The intuition is that the log-likelihood of an encoder's output representations is higher on the victim's training data than on test data if it is stolen from the victim, but not if it is independently trained. We compute this log-likelihood using density estimation models. As part of our evaluation, we also propose measuring the fidelity of stolen encoders and quantifying the effectiveness of the theft detection without involving downstream tasks; instead, we leverage mutual information and distance measurements. Our extensive empirical results in the vision domain demonstrate that dataset inference is a promising direction for defending self-supervised models against model stealing.


page 1

page 2

page 3

page 4


SSL-Cleanse: Trojan Detection and Mitigation in Self-Supervised Learning

Self-supervised learning (SSL) is a commonly used approach to learning a...

On the Difficulty of Defending Self-Supervised Learning against Model Extraction

Self-Supervised Learning (SSL) is an increasingly popular ML paradigm th...

BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning

Self-supervised learning in computer vision aims to pre-train an image e...

ESTAS: Effective and Stable Trojan Attacks in Self-supervised Encoders with One Target Unlabelled Sample

Emerging self-supervised learning (SSL) has become a popular image repre...

SSL-Auth: An Authentication Framework by Fragile Watermarking for Pre-trained Encoders in Self-supervised Learning

Self-supervised learning (SSL), utilizing unlabeled datasets for trainin...

Towards Understanding How Self-training Tolerates Data Backdoor Poisoning

Recent studies on backdoor attacks in model training have shown that pol...

Social Processes: Self-Supervised Forecasting of Nonverbal Cues in Social Conversations

The default paradigm for the forecasting of human behavior in social con...

Please sign up or login with your details

Forgot password? Click here to reset