PhD candidate Vasisht Duddu, Intel Labs research scientist Sebastian Szyller, and Professor N. Asokan have been honoured with a Distinguished Paper Award for their work titled “SoK: Unintended Interactions among Machine Learning Defenses and Risks.” Their paper was presented at the 45th IEEE Symposium on Security and Privacy, the premier forum for showcasing developments in computer security and electronic privacy.
“Congratulations to Vasisht, Asokan, and their colleague Sebastian on receiving a distinguished paper award,” said Raouf Boutaba, University Professor and Director of the Cheriton School of Computer Science. “Although considerable research has been conducted on security and privacy risks in machine learning models, further work is needed to understand how specific defences interact with other risks. Their award-winning systematization of knowledge paper offers a framework to identify and explain interactions between defences and risks, allowing them to conjecture about unintended interactions.”
L to R: Professor N. Asokan and PhD candidate Vasisht Duddu. Sebastian Szyller was unavailable for the photo.
Vasisht Duddu is pursuing a PhD at the Cheriton School of Computer Science. His research focuses on risks to security, privacy, fairness, and transparency in machine learning models. He also designs attacks to exploit these risks and defences to counter them to better understand the interplay between risks and defences. Additionally, he works on ensuring accountability in machine learning pipelines to meet regulatory requirements.
N. Asokan is a Professor of Computer Science at Waterloo where he holds a David R. Cheriton Chair and serves as the Executive Director of the Waterloo Cybersecurity and Privacy Institute. Asokan’s primary research theme is systems security broadly, including topics like developing and using novel platform security features, applying cryptographic techniques to design secure protocols for distributed systems, applying machine learning techniques to security and privacy problems, and understanding and addressing the security and privacy of machine learning applications themselves.
Sebastian Szyller is a research scientist at Intel Labs. He works on various aspects of security and privacy of machine learning. Recently, he has been working on model extraction attacks and defences, membership inference, and differential privacy. More broadly, he is interested in different ways to protect machine learning models and data to enable robust and privacy-preserving analysis, both in terms of the technical details as well as legislation compliance. Sebastian was a visiting PhD student at Waterloo during fall 2022.
More about this award-winning research
Machine learning models are vulnerable to a variety of risks to their security, privacy and fairness. Although various defences have been proposed to protect against risks individually, when a defence is effective against one risk it may inadvertently lead to an increased susceptibility to other risks. Adversarial training to increase robustness, for example, also increases vulnerability to membership inference attacks.
Predicting such unintended interactions is challenging. A unified framework that clarifies the relationship between defences and risks can help researchers identify unexplored interactions and design algorithms with better trade-offs. It also helps practitioners to account for such interactions before deployment. Earlier work, however, was limited to studying a specific risk, defence or interaction as opposed to systematically studying their underlying causes. A comprehensive framework spanning multiple defences and risks to systematically identify potential unintended interactions is currently absent.
The research team addressed this gap by systematically examining various unintended interactions across multiple defences and risks. They hypothesized that overfitting and memorization of training data are the potential causes underlying these unintended interactions. An effective defence may induce, reduce or depend on overfitting or memorization, which in turn affects the model’s susceptibility to other risks.
Their study identified several factors — such as the characteristics of the training dataset, its objective function, and the model — that collectively influence a model’s propensity to overfit or memorize. These factors provide insight into understanding the susceptibility to different risks when a defence is employed.
Key contributions of the study
- Developed the first systematic framework to understand unintended interactions by their underlying causes and factors that influence them
- Conducted a comprehensive literature survey to identify different unintended interactions, situating them within the framework, and a guideline to the framework to hypothesize about unintended interactions
- Identified previously unexplored unintended interactions for future research, using the framework to hypothesize two such interactions and empirically validating them
For further details about this award-winning research, please see the paper: Vasisht Duddu, Sebastian Szyller, N. Asokan. SoK: Unintended Interactions among Machine Learning Defenses and Risks, 2024 IEEE Symposium on Security and Privacy, San Francisco, CA, 2024.
Research group’s project page: https://ssg-research.github.io/mlsec/interactions
Blog article: https://blog.ssg.aalto.fi/2024/05/unintended-interactions-among-ml.html
