Waterloo researchers advancing cybersecurity in the age of AI

Every day, Canadians trust technology with more of their lives, from banking and health information to workplace communications and smart home devices. Increasingly, artificial intelligence (AI) sits behind those interactions.

As AI systems become more capable, there is concern that new security and privacy risks are emerging just as quickly. At the University of Waterloo, researchers are exploring how to protect the next generation of AI-powered technologies from misuse, manipulation and cyberattacks before they reach millions of users. 

"We might not be fully aware that it's everywhere," says Dr. Meng Xu, professor in the Cheriton School of Computer Science in the Faculty of Mathematics and member of the Cybersecurity and Privacy Institute (CPI). "Cybersecurity is basically everywhere that's connected to our lives."

As Canada marks Cybersecurity Awareness Month, Waterloo researchers are investigating some of the newest challenges emerging alongside artificial intelligence. Among them are Xu and fellow professor Dr. Sihang Liu, whose work focuses on identifying vulnerabilities in AI systems before they become widespread problems.

One area of growing concern is AI agents, systems capable of independently completing tasks by combining large language models with external tools and software. While these systems can boost productivity, they also introduce new security risks.

Unlike traditional AI systems, AI agents can take actions on a user's behalf, such as accessing information or interacting with software. That autonomy creates new opportunities for attackers to manipulate their behaviour.

"It really becomes an arms race on how well the defender can use this tool to build secure, more reliable software versus how the attacker can use this tool to find holes and penetrate the system," Xu says.

"How can we make sure this is safe?" Liu says, noting that AI systems can generate outputs at a scale that is difficult for people to review manually.

To help address these challenges, Xu and Liu received support from the National Cybersecurity Consortium (NCC) for research exploring how AI agents can be protected from malicious or compromised tools. The research reflects the broader mission of CPI, where researchers from all six faculties, from computer science and engineering to sociology, law, health and economics collaborate on increasingly complex cybersecurity challenges. Both researchers say CPI’s interdisciplinary approach is one of Waterloo's greatest strengths.

Waterloo is one of Canada's leading cybersecurity hubs because it brings together a unique combination of research excellence, interdisciplinary collaboration, industry partnerships and talent development.

According to Xu, Waterloo students demonstrate an unusually strong cybersecurity mindset, reinforced through specialized courses, competitions and co-op experiences.

The University's extensive co-op network also helps students apply security knowledge directly in workplaces and bring real-world challenges back into the classroom. Together with a growing community of faculty and graduate researchers, that creates an ecosystem capable of addressing both today's threats and the ones that will continue to emerge in the future.

Despite the rapid pace of technological change, both researchers remain optimistic. While AI adoption will continue to introduce new security challenges, they believe research, education and better system design will help build safer and more resilient technologies.

"There will definitely be more solutions than problems as the tech continues to change,” Liu says. “These challenges will eventually be solved as we develop more reliable, trustworthy AI systems and build security into their design from the start.”

As that work continues at Waterloo, Xu and Liu are focusing on anticipating tomorrow's risks, training the next generation of cybersecurity leaders and helping ensure that the technologies shaping Canada's future remain secure and trustworthy.