Nils Lukas, a PhD candidate at the Cheriton School of Computer Science, is the first-place winner of the 2024 Faculty of Mathematics Doctoral Prize. Now in its sixth year, this prestigious award recognizes and celebrates the achievements of top doctoral students in the Faculty of Mathematics. As a first-place recipient, Nils will receive $1,500 and is nominated for the university-wide Governor General’s Gold Medal, which is awarded at spring convocation.
“Congratulations to Nils on receiving this prestigious and much-deserved recognition,” said Raouf Boutaba, University Professor and Director of the Cheriton School of Computer Science. “The research he has conducted on trustworthy, secure and privacy-preserving machine learning and published at the top international conferences in these fields is not only academically rigorous but also hugely significant for industry and society.”
Nils works on most of the pressing security and privacy problems in machine learning — untrustworthy data, untrustworthy providers, and untrustworthy users, explains his advisor Professor Florian Kerschbaum.
“Across these areas Nils has an outstanding publication record and at a level of academic excellence rarely seen among even applicants for faculty positions,” Professor Kerschbaum said. “Since joining my group, Nils has published five first-author papers and another as the supervisor of an undergraduate student, all of them in the top venues, with several other papers in submission. His published works include his paper presented at the IEEE Symposium on Security and Privacy in 2022 and another in 2023, a paper presented at USENIX Security Symposium in 2023, a paper in the International Conference on Learning Representations in 2021, followed by two more at that venue in 2024.”
Nils
Lukas,
a
PhD
candidate
in
the
Cheriton
School
of
Computer
Science’s
Cryptography,
Security,
and
Privacy
(CrySP)
group,
focuses
on
trustworthy
machine
learning.
He
has
an
MSc
with
distinction
in
Computer
Science
from
RWTH
Aachen
University
in
Germany.
His
research
explores
the
threats
that
arise
when
deploying
deep
neural
networks
from
three
perspectives:
(1)
privacy
when
the
model
is
trained
on
private
data,
(2)
reliability
when
the
model’s
training
data
cannot
be
trusted,
and
(3)
model
misuse
when
the
users
cannot
be
trusted.
His
work
includes
studying
privacy
attacks
against
large
language
models
fine-tuned
on
private
datasets,
developing
defences
against
data
poisoning,
and
creating
multiple
methods
for
controlling
model
misuse.
Nils
received
a
prestigious
2022–24
Cheriton
Graduate
Scholarship.
Additionally,
his
research
has
won
two
notable
poster
competitions:
the
2023
Cheriton
Research
Symposium
poster
competition
and
the
2019
Cybersecurity
and
Privacy
Institute
poster
competition.
More about Nils Lukas’s research
The rapid advancement of generative AI models in recent years holds great promise to transform businesses and society, but they also pose novel trust, security and privacy challenges. The research Nils conducts is helping to reduce the risks of these technologies.
In his paper titled Analyzing Leakage of Personally Identifiable Information in Language Models, published in IEEE Symposium on Security & Privacy in 2023 with colleagues from Microsoft Research, Nils introduced novel attack algorithms capable of extracting ten times more personally identifiable information than existing attacks. This work revealed that standard sentence-level differentially private training, while largely reducing the risk of disclosing personally identifiable information, still leaks about 3% of such information. The significance of this work is that it is one of the first comprehensive studies of the risk of personally identifiable information memorization in language models, and it exposed the subtle insufficiency of sentence-level differentially private training for protecting record level personally identifiable information. Nils has released his code to the public to reproduce and conduct further research.
In SoK: How Robust is Image Classification Deep Neural Network Watermarking?, a paper with Edward Jiang, Xinda Li and Florian Kerschbaum presented at IEEE Symposium on Security & Privacy in 2022, Nils conducted a systematic evaluation of the robustness of existing watermarking schemes that aim to verify provenance of machine learning models and to prevent misuse of AI generated content. Nils found that none of the surveyed watermarking schemes can withstand all removal attacks, showcasing the importance of a thorough evaluation framework.
In Deep Neural Network Fingerprinting by Conferrable Adversarial Examples, a paper with Yuxuan Zhang and Florian Kerschbaum presented at ICLR 2021, Nils developed a fingerprinting method for deep neural networks, aimed at detecting the surrogate models that an adversary may build by querying a proprietary source model. Nils proposed a new method to generate conferrable adversarial examples and, importantly, demonstrated their superior effectiveness and robustness against previous fingerprints and watermarks.
In PTW: Pivotal Tuning Watermarking for Pre-Trained Image Generators, a paper with his advisor presented at the 32nd USENIX Security Symposium, Nils explored image generators, such as those used in deepfake creation. He proposed pivotal tuning watermarking to prevent misuse of image generators, achieving three orders of magnitude speedup while obviating the need of any training data. Moreover, Nils revealed some intrinsic trade-off between the undetectability and robustness of watermarks.
In Leveraging Optimization for Adaptive Attacks on Image Watermarks, a paper with Abdulrahman Diaa, Lucas Fenaux, and Florian Kerschbaum presented at ICLR 2024, the authors continued the investigation of image watermarking attacks through the lens of adaptive, learnable attacks. The core idea is that an adaptive attacker who knows the watermarking algorithm can create their own surrogate keys and use them to optimize the parameters of a watermark removal attack. Such adaptive, learnable attacks can undermine the robustness of all five tested, state-of-the-art watermarking methods and require limited computational resources. Nils has presented his watermarking results to Google, with the goal that the research will limit misuse of its image generators and combat misinformation.