Adversarial Machine Learning | Security Research Group

Microsoft Research Summit: Surprising (and unsurprising) Inference Risks in Machine Learning

21 October 2021 adversarial machine learning, privacy, Bargav Jayaraman, Anshuman Suri, Katherine Knipmeyer, inference privacy, privacy, privacy-preserving machine learning, Microsoft

Here are the slides for my talk at the Practical and Theoretical Privacy of Machine Learning Training Pipelines Workshop at the Microsoft Research Summit (21 October 2021): Surprising (and Unsurprising) Inference Risks in Machine Learning [PDF] The work by Bargav Jayaraman (with Katherine Knipmeyer, Lingxiao Wang, and Quanquan Gu) that I talked about on improving membership inference attacks is described in more details here: Bargav Jayaraman, Lingxiao Wang, Katherine Knipmeyer, Quanquan Gu, David Evans.

UVA News Article

28 September 2021 adversarial machine learning, privacy, Bargav Jayaraman, Xiao Zhang, Jack Prescott, Anshuman Suri, Katherine Knipmeyer, inference privacy, privacy, privacy-preserving machine learning

UVA News has an article by Audra Book on our research on security and privacy of machine learning (with some very nice quotes from several students in the group, and me saying something positive about the NSA!): Computer science professor David Evans and his team conduct experiments to understand security and privacy risks associated with machine learning, 8 September 2021. David Evans, professor of computer science in the University of Virginia School of Engineering and Applied Science, is leading research to understand how machine learning models can be compromised.

Model-Targeted Poisoning Attacks with Provable Convergence

29 June 2021 adversarial machine learning, model-targeted poisoning attacks, Fnu Suya, Saeed Mahloujifar, Anshuman Suri, David Evans, Yuan Tian, ICML 2021

(Post by Sean Miller, using images adapted from Suya’s talk slides) Data Poisoning Attacks Machine learning models are often trained using data from untrusted sources, leaving them open to poisoning attacks where adversaries use their control over a small fraction of that training data to poison the model in a particular way. Most work on poisoning attacks is directly driven by an attacker’s objective, where the adversary chooses poisoning points that maximize some target objective.

ICLR DPML 2021: Inference Risks for Machine Learning

7 May 2021 adversarial machine learning, privacy, Bargav Jayaraman, Anshuman Suri, Katherine Knipmeyer, inference privacy, privacy, privacy-preserving machine learning

I gave an invited talk at the Distributed and Private Machine Learning (DPML) workshop at ICLR 2021 on Inference Risks for Machine Learning.

The talk mostly covers work by Bargav Jayaraman on evaluating privacy in machine learning and connecting attribute inference and imputation, and recent work by Anshuman Suri on property inference.

How to Hide a Backdoor

6 May 2021 adversarial machine learning, research, Fnu Suya, Yulong Tian

The Register has an article on our recent work on Stealthy Backdoors as Compression Artifacts: Thomas Claburn, How to hide a backdoor in AI software — Neural networks can be aimed to misbehave when squeezed, The Register, 5 May 2021.

Codaspy 2021 Keynote: When Models Learn Too Much

26 April 2021 adversarial machine learning, privacy, Bargav Jayaraman, Anshuman Suri, Katherine Knipmeyer, inference privacy, privacy, privacy-preserving machine learning

Here are the slides for my talk at the 11th ACM Conference on Data and Application Security and Privacy: When Models Learn Too Much [PDF] The talk includes Bargav Jayaraman’s work (with Katherine Knipmeyer, Lingxiao Wang, and Quanquan Gu) on evaluating privacy in machine learning, as well as more recent work by Anshuman Suri on property inference attacks, and Bargav on attribute inference and imputation: Merlin, Morgan, and the Importance of Thresholds and Priors Evaluating Differentially Private Machine Learning in Practice “When models learn too much.

CrySP Talk: When Models Learn Too Much

5 April 2021 adversarial machine learning, privacy-preserving machine learning, differential privacy, videos

I gave a talk on When Models Learn Too Much at the University of Waterloo (virtually) in the CrySP Speaker Series on Privacy (29 March 2021): Abstract Statistical machine learning uses training data to produce models that capture patterns in that data. When models are trained on private data, such as medical records or personal emails, there is a risk that those models not only learn the hoped-for patterns, but will also learn and expose sensitive information about their training data.

Improved Estimation of Concentration (ICLR 2021)

2 April 2021 adversarial machine learning, Jack Prescott, Xiao Zhang, intrinsic robustness, ICLR

Our paper on Improved Estimation of Concentration Under ℓp-Norm Distance Metrics Using Half Spaces (Jack Prescott, Xiao Zhang, and David Evans) will be presented at ICLR 2021. Abstract: Concentration of measure has been argued to be the fundamental cause of adversarial vulnerability. Mahloujifar et al. (2019) presented an empirical way to measure the concentration of a data distribution using samples, and employed it to find lower bounds on intrinsic robustness for several benchmark datasets.

Microsoft Security Data Science Colloquium: Inference Privacy in Theory and Practice

1 December 2020 adversarial machine learning, privacy, Bargav Jayaraman, privacy, privacy-preserving machine learning, Microsoft

Here are the slides for my talk at the Microsoft Security Data Science Colloquium:
When Models Learn Too Much: Inference Privacy in Theory and Practice [PDF]

The talk is mostly about Bargav Jayaraman’s work (with Katherine Knipmeyer, Lingxiao Wang, and Quanquan Gu) on evaluating privacy:

Adversarially Robust Representations

14 August 2020 adversarial machine learning, ICML, Sicheng Zhu, Xiao Zhang

Post by Sicheng Zhu With the rapid development of deep learning and the explosive growth of unlabeled data, representation learning is becoming increasingly important. It has made impressive applications such as pre-trained language models (e.g., BERT and GPT-3). Popular as it is, representation learning raises concerns about the robustness of learned representations under adversarial settings. For example, how can we compare the robustness to different representations, and how can we build representations that enable robust downstream classifiers?

All Posts by Category or Tags.