When Relaxations Go Bad: "Differentially-Private" Machine Learning

by   Bargav Jayaraman, et al.

Differential privacy is becoming a standard notion for performing privacy-preserving machine learning over sensitive data. It provides formal guarantees, in terms of the privacy budget, ϵ, on how much information about individual training records is leaked by the model. While the privacy budget is directly correlated to the privacy leakage, the calibration of the privacy budget is not well understood. As a result, many existing works on privacy-preserving machine learning select large values of ϵ in order to get acceptable utility of the model, with little understanding of the concrete impact of such choices on meaningful privacy. Moreover, in scenarios where iterative learning procedures are used which require privacy guarantees for each iteration, relaxed definitions of differential privacy are often used which further tradeoff privacy for better utility. In this paper, we evaluate the impacts of these choices on privacy in experiments with logistic regression and neural network models. We quantify the privacy leakage in terms of advantage of the adversary performing inference attacks and by analyzing the number of members at risk for exposure. Our main findings are that current mechanisms for differential privacy for machine learning rarely offer acceptable utility-privacy tradeoffs: settings that provide limited accuracy loss provide little effective privacy, and settings that provide strong privacy result in useless models.


page 1

page 2

page 3

page 4


LinkedIn's Audience Engagements API: A Privacy Preserving Data Analytics System at Scale

We present a privacy system that leverages differential privacy to prote...

Every Query Counts: Analyzing the Privacy Loss of Exploratory Data Analyses

An exploratory data analysis is an essential step for every data analyst...

That which we call private

A casual reader of the study by Jayaraman and Evans in USENIX Security 2...

Quantifying identifiability to choose and audit ε in differentially private deep learning

Differential privacy allows bounding the influence that training data re...

Tight Auditing of Differentially Private Machine Learning

Auditing mechanisms for differential privacy use probabilistic means to ...

P4L: Privacy Preserving Peer-to-Peer Learning for Infrastructureless Setups

Distributed (or Federated) learning enables users to train machine learn...

Differential Privacy at Risk: Bridging Randomness and Privacy Budget

The calibration of noise for a privacy-preserving mechanism depends on t...

Please sign up or login with your details

Forgot password? Click here to reset