papers | Security Research Group

SaTML Talk: SoK: Pitfalls in Evaluating Black-Box Attacks

22 April 2024 Fnu Suya, Anshuman Suri, Tingwei Zhang, Jingtao Hong, Yuan Tian, David Evans, SaTML, adversarial machine learning, black-box adversarial attacks, systemization of knowledge

Anshuman Suri’s talk at IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) is now available:

See the earlier blog post for more on the work, and the paper at https://arxiv.org/abs/2310.17534.

Do Membership Inference Attacks Work on Large Language Models?

5 March 2024 Anshuman Suri, adversarial machine learning, privacy-preserving machine learning, distribution inference, inference privacy, LLMs, Michael Duan, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, Hannaneh Hajishirzi

MIMIR logo. Image credit: GPT-4 + DALL-E Paper Code Data Membership inference attacks (MIAs) attempt to predict whether a particular datapoint is a member of a target model’s training data. Despite extensive research on traditional machine learning models, there has been limited work studying MIA on the pre-training data of large language models (LLMs). We perform a large-scale evaluation of MIAs over a suite of language models (LMs) trained on the Pile, ranging from 160M to 12B parameters.

SoK: Pitfalls in Evaluating Black-Box Attacks

20 December 2023 Fnu Suya, Anshuman Suri, Tingwei Zhang, Jingtao Hong, Yuan Tian, David Evans, SaTML, adversarial machine learning, black-box adversarial attacks, systemization of knowledge

Post by Anshuman Suri and Fnu Suya Much research has studied black-box attacks on image classifiers, where adversaries generate adversarial examples against unknown target models without having access to their internal information. Our analysis of over 164 attacks (published in 102 major security, machine learning and security conferences) shows how these works make different assumptions about the adversary’s knowledge. The current literature lacks cohesive organization centered around the threat model. Our SoK paper (to appear at IEEE SaTML 2024) introduces a taxonomy for systematizing these attacks and demonstrates the importance of careful evaluations that consider adversary resources and threat models.

NeurIPS 2023: What Distributions are Robust to Poisoning Attacks?

7 December 2023 Xiao Zhang, Fnu Suya, adversarial machine learning, indiscriminate poisoning attacks, susceptibility variation

Post by Fnu Suya Data poisoning attacks are recognized as a top concern in the industry [1]. We focus on conventional indiscriminate data poisoning attacks, where an adversary injects a few crafted examples into the training data with the goal of increasing the test error of the induced model. Despite recent advances, indiscriminate poisoning attacks on large neural networks remain challenging [2]. In this work (to be presented at NeurIPS 2023), we revisit the vulnerabilities of more extensively studied linear models under indiscriminate poisoning attacks.

SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning

26 May 2023 Ahmed Salem, Anshuman Suri, Giovanni Cherubin, Boris Köpf, Andrew Paverd, Shruti Tople, Santiago Zanella-Béguelin, adversarial machine learning, privacy-preserving machine learning, membership inference, inference privacy

Our paper on the use of cryptographic-style games to model inference privacy is published in IEEE Symposium on Security and Privacy (Oakland):

Giovanni Cherubin, , Boris Köpf, Andrew Paverd, Anshuman Suri, Shruti Tople, and Santiago Zanella-Béguelin. SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning. IEEE Symposium on Security and Privacy, 2023. [Arxiv]

Tired of diverse definitions of machine learning privacy risks? Curious about game-based definitions? In our paper, we present privacy games as a tool for describing and analyzing privacy risks in machine learning. Join us on May 22nd, 11 AM @IEEESSP '23 https://t.co/NbRuTmHyd2 pic.twitter.com/CIzsT7UY4b
— ahmed salem (@AhmedGaSalem) May 15, 2023

CVPR 2023: Manipulating Transfer Learning for Property Inference

2 May 2023 Yulong Tian, Anshuman Suri, Fnu Suya, adversarial machine learning, privacy-preserving machine learning, distribution inference, inference privacy, transfer learning

Manipulating Transfer Learning for Property Inference Transfer learning is a popular method to train deep learning models efficiently. By reusing parameters from upstream pre-trained models, the downstream trainer can use fewer computing resources to train downstream models, compared to training models from scratch. The figure below shows the typical process of transfer learning for vision tasks: However, the nature of transfer learning can be exploited by a malicious upstream trainer, leading to severe risks to the downstream trainer.

Attribute Inference attacks are really Imputation

7 December 2022 Bargav Jayaraman, privacy-preserving machine learning, attribute inference, distribution inference

Post by Bargav Jayaraman Attribute inference attacks have been shown by prior works to pose privacy threat against ML models. However, these works assume the knowledge of the training distribution and we show that in such cases these attacks do no better than a data imputataion attack that does not have access to the model. We explore the attribute inference risks in the cases where the adversary has limited or no prior knowledge of the training distribution and show that our white-box attribute inference attack (that uses neuron activations to infer the unknown sensitive attribute) surpasses imputation in these data constrained cases.

All Posts by Category or Tags.