3D Vision for robot planning
As a robot manipulates 3D objects and navigates 3D scenes, it requires spatial reasoning to ensure safe planning. Recent advances in 3D scene representation, such as Gaussian Splatting, enable the creation of high-fidelity digital twins of real-world environments from multi-view images. This project leverages 3D vision-language fields for open-vocabulary robot planning. (Knowledge of robotics, machine learning, and tools such as PyTorch is required.)
Supervisor: Roya Firoozi
Email: roya.firoozi@uwaterloo.ca
Accelerated Reinforcement Learning Policy Development for Industrial Robot Manipulation Using NVIDIA Isaac Lab
This project investigates model-free and policy-optimization reinforcement learning within NVIDIA Isaac Lab to accelerate skill acquisition for robotic manipulators. Students will optimize sample efficiency and convergence rates through advanced reward shaping, discrete/continuous action-space formulations, and curriculum learning paradigms, ultimately establishing a robust, repeatable sim-to-real workflow for adaptive industrial manufacturing systems.
Required Knowledge & Skills: Familiarity with reinforcement learning fundamentals, including policies, rewards, observations, actions, and exploration; proficiency in Python and PyTorch; experience with NVIDIA Isaac Sim, NVIDIA Isaac Lab, or similar robotic simulation environments; understanding of robotic manipulators, coordinate systems, kinematics, and gripper control; experience with reinforcement learning frameworks such as RSL-RL, RL-Games, or SKRL is an asset; familiarity with reward shaping, curriculum learning, domain randomization, and sim-to-real concepts is beneficial.
Application Instructions: Interested applicants should combine their CV, undergraduate transcripts, and graduate transcripts into a single PDF file and email it directly to the supervisor. This project is conducted in collaboration with an industry partner, offering strong potential for future co-op opportunities.
Supervisor: Professor Ladan Tahvildari
AI for Engineering – Enhancing Nonlinear Approximation with Learnable Activation Functions
Artificial Intelligence (AI) systems, particularly those based on deep neural networks (DNNs), can be interpreted as nonlinear function approximators. The expressive power of AI in engineering applications largely stems from its capacity to handle complex nonlinear relationships. In current DNN architectures, this nonlinearity is primarily introduced through fixed activation functions such as ReLU, Sigmoid, Tanh, GELU, Swish, and Leaky ReLU. While these functions are simple and efficient, they offer limited flexibility and nonlinear expressiveness. As a result, large-scale DNNs with many layers and parameters are often required to model intricate nonlinear phenomena, leading to increased computational cost and training complexity, and often without significantly improving approximation quality.
To address these limitations, we have recently developed an analytic, learnable activation function capable of approximating any continuous nonlinear function to arbitrary precision. This development opens the door to a new class of compact, high-capacity neural networks with greatly enhanced nonlinear modeling power. When integrated with mathematical frameworks such as the Kolmogorov Representation Theorem, this novel activation mechanism enables the design of small-scale yet powerful DNNs that can approximate complex engineering functions that are otherwise intractable or require prohibitively large models.
Leveraging our learnable activation functions, this project aims to develop a new AI framework tailored for engineering applications, where more efficient, compact, and theoretically grounded AI systems can be built for engineering design, control, and optimization.
Supervisor: Prof. En-Hui Yang
Email: ehyang@uwaterloo.ca
Building a Radar System
The goal of this course project is to help students understand basic radar concepts. The course spans topics of applied electromagnetics, antennas, RF design, analog circuits, digital signal processing, machine learning, and artificial intelligence. Students will decide upon a radar application (whether for autonomous drones/robots/vehicles or in the general theme of sensing for healthcare). Students will then get to work towards building a simulation model of their own radar system.
Supervisor: Prof. G. Shaker (Adjunct)
Email: gshaker@uwaterloo.ca
Causal Inference for Simulation-Based Decision Making
This project focuses on improving causal reasoning in AI systems. This could include improving causal inference via learning from simulated or real-world data. Alternatively, it could involve looking at failures of existing generative models for text or images in causal reasoning
Required Knowledge & Skills:
Prior experience with machine learning experimental methods, and implementing some deep learning models in Python-based frameworks, is expected.
Supervisor: Prof. Mark Crowley
Email: mark.crowley@uwaterloo.ca
Consistency Learning in Forest LiDAR Data
This project investigates the development of novel machine learning architectures tailored to structured spatial data, with a focus on LiDAR scans of forest environments. The student will explore how prior domain knowledge and structural inductive biases can improve learning performance, robustness, and generalization in models trained on complex 3D sensor data.
Required Knowledge & Skills:
Prior experience with machine learning experimental methods, and implementing some deep learning models in Python-based frameworks, is expected.
Supervisor: Prof. Mark Crowley
Email: mark.crowley@uwaterloo.ca
Detecting Adversarial Examples for Deep Neural Networks through Gradient
It is hypothesized that the gradient map of a benign image will be very different from that of a corresponding adversarial example in terms of their statistic properties, spatial shapes, and visualization. This project will first analyze the differences between the gradient maps of benign images and the gradient maps of corresponding adversarial examples. Based on the analysis, the project will then explore novel ways to detect adversarial examples through gradients.
Supervisor: Prof. En-Hui Yang
Email: ehyang@uwaterloo.ca
Ethical Reasoning in Reinforcement Learning Agents
This project explores how reinforcement learning agents can be designed to exhibit ethical decision-making in environments that simulate moral dilemmas. The student will investigate frameworks for embedding moral principles or meta-rules into learning systems and evaluate agent behavior in simulated contexts such as Minecraft.
Required Knowledge & Skills:
Prior experience with machine learning experimental methods, Deep Reinforcement Learning, and implementing some deep learning models in Python-based frameworks, is expected.
Supervisor: Prof. Mark Crowley
Email: mark.crowley@uwaterloo.ca
LLM-Assisted Production Intelligence and Closed-Loop Control via Digital Twin Environments
This project develops an LLM-based intelligent assistant integrated with a virtual digital twin to streamline industrial automation control. By combining synthetic operational data streams with technical documentation and SOPs, the system analyzes trends, diagnoses anomalies, and executes closed-loop parameter adjustments to significantly reduce operator cognitive load and optimize overall manufacturing performance.
Required knowledge & skills: Familiarity with large language models and prompt engineering; experience with retrieval-augmented generation (RAG) and document-grounded QA systems; understanding of automation systems, PLC/SCADA concepts, and production workflows; proficiency in Python; experience with data analysis and time-series production data; knowledge of API integration and orchestration frameworks (e.g., LangChain, LlamaIndex, or similar); familiarity with digital twin platforms is an asset.
Application Instructions: Interested applicants should combine their CV, undergraduate transcripts, and graduate transcripts into a single PDF file and email it directly to the supervisor. This project is conducted in collaboration with an industry partner, offering strong potential for future co-op opportunities.
Supervisor: Professor Ladan Tahvildari
Modelling Adversarial Perturbations in Deep Neural Networks
Deep neural networks (DNNs) are vulnerable to adversarial examples, maliciously modified raw input data which is imperceptible to human vision, but once fed into DNNs, can lead DNNs to produce incorrect outputs. The existence and easy construction of adversarial examples pose significant security risks to DNNs, especially in safety-critical applications, including visual object recognition and autonomous driving. The objective of this project is to model adversarial perturbations in DNNs through statistical analysis and DNN visualization. The established model will provide a basis for developing a radically different approach for detecting adversarial examples.
Supervisor: Prof. En-Hui Yang
Email: ehyang@uwaterloo.ca
Perceptually motivated and deep learning approaches for image and video processing
The objective of this project is to develop novel methodologies for image and video processing, optimization, compression, transmission, and streaming based on advanced technologies including perceptually motivated and deep learning approaches. Working with a group of experienced researchers and fellow students, the student will carry out research in the forms of algorithm and software development, experiment design and setup, perceptual testing, and data processing and analysis
Supervisor: Prof. Zhou Wang
Email: zhou.wang@uwaterloo.ca
Robot learning to adapt to user preferences
Robots are being deployed in human-centric environments, working alongside and with humans. However, different people have different preferences on how a robot should act -- how fast it should move, how close it can come, and how it should interact. These preferences are user specific, and so the robot should learn them online, and adapt in real-time. This project will build on our recent work to develop learning algorithms for robots to adapt to human preferences and improve robot performance.
Supervisor: Prof. Stephen L. Smith
Email: stephen.smith@uwaterloo.ca
Project:
Self-driving vehicles are becoming reality. Thanks to technological advances in communication and in artificial intelligence, self-driving vehicles are becoming a much closer reality than we think. Indeed, recognizing the undisputed promise of self-driving to improve safety and road efficiency, several cities around the globe have started to conduct self-driving trials in order to ensure their readiness for deploying this revolutionary technology. Nonetheless, self-driving vehicles currently face several challenges: They are limited in their speed performance, crash prevention, sensing, cooperation, and coordination capabilities. Their decision making, and hence their actions, rely merely on their on-board sensory and internal control models. To avoid crashes and prevent traffic congestion, self-driving vehicles must anticipate the behavior of other vehicles in their environment (self-driving and human-operated vehicles), share their internal state with these vehicles, and cooperatively operate to choose joint safe and efficient control policies. A primitive example of self-driving cooperative behavior can be seen in platooning, where a small group of trucks assemble in a linear structure, using wireless connectivity to maintain a prescribed distance between each other. However, more research is needed to investigate and develop techniques that enable full autonomy performance in irregular vehicle arrangements such as convoys, and in unstructured driving situations. Moving from structured small-scale working environments to large-scale unstructured working environments presents a major challenge to the future of self- driving technology due to the lack of proper understanding and modeling of the interaction dynamics in such situations; as well as due to the lack of collective sensing and cooperating methodologies that can facilitate cooperation in complex driving situations. Such collaboration is potentially feasible, given the emerging communication technology (e.g., IEEE 802.11p standard) and V2X communications services under the future 5G wireless standard. In this context, the goal of this research program is the development of a framework for coordinating interaction and cooperation among a convoy of self-driving vehicles, operating in complex driving conditions in the presence of human-operated vehicles.
1. Behaviour Modeling of Human-Driven Vehicles
Develop a class of models that can mimic the behavior of human-operated vehicles under various driving conditions- environmental, infrastructure, and traffic, and for mitigating the impact of human-operated vehicles on self-driving performance, in general, and safety in particular.
2. Situation Assessment in Human/Machine Driven Environments
Develop strategies for situational assessment, both at the vehicle level and the convoy level.
3. Self-Driving Cooperation Strategies
Develop tactical self-driving strategies that can enable interaction and cooperation between self-driving vehicles to facilitate joint mobility control and coordination, in the presence of human-operated vehicles.
4. Cooperative Resource Sharing in Self-Driving Applications
Develop self-driving strategies that can enable cooperation between convoys (coalitions) of self-driving vehicles to facilitate optimal trip planning and infrastructure resources sharing.
Note: Various group/collective behavior use-case studies should be conducted to validate the outcome of each project, both theoretically and using simulations. Examples of use cases of interest include speed harmonization, emergency response, intersection negotiation, crash avoidance, platoons and Convoys.
Towards Developing More Affordable, Efficient, Interpretable, and Secure Artificial Intelligence Paradigms
While deep learning-based artificial intelligence (AI) has propelled the information age to unprecedented heights, its current trajectory—characterized by scaling up massive deep neural networks (DNNs) and leveraging vast datasets—faces significant limitations. These include enormous demands for energy, computing resources, and high-throughput networking, alongside persistent challenges in AI interpretability and security. Moreover, the high costs associated with this approach restrict accessibility to a few large players, limiting its societal impact and potential to create broader economic opportunities.
To address these challenges, this project aims to explore and develop alternative AI paradigms that prioritize efficiency, interpretability, security, and inclusivity. By leveraging concepts and techniques from information science and probability theory, the project will initially focus on enhancing the nonlinearity, structural properties, and training/inference efficiencies of DNNs, a critical step towards achieving the final goal.
Supervisor: Prof. En-Hui Yang
Email: ehyang@uwaterloo.ca