privacy-preserving machine learning | Security Research Group

CrySP Talk: When Models Learn Too Much

5 April 2021 adversarial machine learning, privacy-preserving machine learning, differential privacy, videos

I gave a talko on When Models Learn Too Much at the University of Waterloo (virtually) in the CrySP Speaker Series on Privacy (29 March 2021): Abstract Statistical machine learning uses training data to produce models that capture patterns in that data. When models are trained on private data, such as medical records or personal emails, there is a risk that those models not only learn the hoped-for patterns, but will also learn and expose sensitive information about their training data.

Microsoft Security Data Science Colloquium: Inference Privacy in Theory and Practice

1 December 2020 adversarial machine learning, privacy, Bargav Jayaraman, privacy, privacy-preserving machine learning

Here are the slides for my talk at the Microsoft Security Data Science Colloquium:
When Models Learn Too Much: Inference Privacy in Theory and Practice [PDF]

The talk is mostly about Bargav Jayaraman’s work (with Katherine Knipmeyer, Lingxiao Wang, and Quanquan Gu) on evaluating privacy:

Merlin, Morgan, and the Importance of Thresholds and Priors

2 October 2020 privacy, privacy-preserving machine learning, Bargav Jayaraman, Lingxiao Wang, Katherine Knipmeyer, Quanquan Gu

Post by Katherine Knipmeyer Machine learning poses a substantial risk that adversaries will be able to discover information that the model does not intend to reveal. One set of methods by which consumers can learn this sensitive information, known broadly as membership inference attacks, predicts whether or not a query record belongs to the training set. A basic membership inference attack involves an attacker with a given record and black-box access to a model who tries to determine whether said record was a member of the model’s training set.

Evaluating Differentially Private Machine Learning in Practice

27 August 2019 privacy, privacy-preserving machine learning, Bargav Jayaraman, differential privacy

(Cross-post by Bargav Jayaraman) With the recent advances in composition of differential private mechanisms, the research community has been able to achieve meaningful deep learning with privacy budgets in single digits. Rènyi differential privacy (RDP) is one mechanism that provides tighter composition which is widely used because of its implementation in TensorFlow Privacy (recently, Gaussian differential privacy (GDP) has shown a tighter analysis for low privacy budgets, but it was not yet available when we did this work).

USENIX Security Symposium 2019

26 August 2019 privacy, privacy-preserving machine learning, Bargav Jayaraman, Sam Havron, Serge Egelman

Bargav Jayaraman presented our paper on Evaluating Differentially Private Machine Learning in Practice at the 28th USENIX Security Symposium in Santa Clara, California. Summary by Lea Kissner: Hey it's the results! pic.twitter.com/ru1FbkESho — Lea Kissner (@LeaKissner) August 17, 2019 Also, great to see several UVA folks at the conference including: Sam Havron (BSCS 2017, now a PhD student at Cornell) presented a paper on the work he and his colleagues have done on computer security for victims of intimate partner violence.

How AI could save lives without spilling medical secrets

14 May 2019 privacy-preserving machine learning, multi-party computation, differential privacy, news

I’m quoted in this article by Will Knight focused on the work Oasis Labs (Dawn Song’s company) is doing on privacy-preserving medical data analysis: How AI could save lives without spilling medical secrets, MIT Technology Review, 14 May 2019. “The whole notion of doing computation while keeping data secret is an incredibly powerful one,” says David Evans, who specializes in machine learning and security at the University of Virginia. When applied across hospitals and patient populations, for instance, machine learning might unlock completely new ways of tying disease to genomics, test results, and other patient information.

NeurIPS 2018: Distributed Learning without Distress

8 December 2018 privacy-preserving machine learning, secure computation, Bargav Jayaraman

Bargav Jayaraman presented our work on privacy-preserving machine learning at the 32nd Conference on Neural Information Processing Systems (NeurIPS 2018) in Montreal. Distributed learning (sometimes known as federated learning) allows a group of independent data owners to collaboratively learn a model over their data sets without exposing their private data. Our approach combines differential privacy with secure multi-party computation to both protect the data during training and produce a model that provides privacy against inference attacks.

All Posts by Category or Tags.