Every day, Canadians trust technology with more of their lives, from banking and health information to workplace communications and smart home devices. Increasingly, artificial intelligence sits behind those interactions. As AI systems become more capable, concern is growing that new security and privacy risks are emerging just as quickly.
Cheriton School of Computer Science researchers are exploring how to protect the next generation of AI-powered technologies from misuse, manipulation and cyberattacks before they reach millions of users.
“We might not be fully aware that it’s everywhere,” says Meng Xu, a professor in the Cheriton School of Computer Science. “Cybersecurity is basically everywhere that’s connected to our lives.”
As Canada marks Cybersecurity Awareness Month, Waterloo researchers are investigating some of the newest challenges emerging alongside artificial intelligence. Among them are Professor Xu and fellow Cheriton School of Computer Science Professor Sihang Liu, whose work focuses on identifying vulnerabilities in AI systems before they become widespread problems.
One area of growing concern is AI agents, systems capable of independently completing tasks by combining large language models with external tools and software. While these systems can boost productivity, they also introduce new security risks.
Unlike traditional AI systems, AI agents can take actions on a user’s behalf, such as accessing information or interacting with software. That autonomy creates new opportunities for attackers to manipulate their behaviour.
“It really becomes an arms race on how well the defender can use this tool to build secure, more reliable software versus how the attacker can use this tool to find holes and penetrate the system,” Professor Xu says.
“How can we make sure this is safe?” Professor Liu says, noting that AI systems can generate outputs at a scale that is difficult for people to review manually.
L to R: Professors Meng Xu and Sihang Liu
Meng Xu is an Assistant Professor at the Cheriton School of Computer Science and a member of Waterloo’s Cybersecurity and Privacy Institute. His research focuses on system and software security, with a focus on delivering high-quality solutions to practical security programs, especially in finding and patching vulnerabilities in critical computer systems.
Sihang Liu is an Assistant Professor at the Cheriton School of Computer Science and a member of Waterloo’s Cybersecurity and Privacy Institute. His research interests lie broadly in computer architecture and systems. Specifically, his research builds and optimizes systems for generative AI applications and AI agents.
Read the full article on Waterloo News