Up, up and away with WISA and AI

With Canada in need of airplane pilots, University of Waterloo researchers are using artificial intelligence (AI) to help more of them take flight.

Supported by the Government of Canada, through the Federal Economic Development Agency for Southern Ontario (FedDev Ontario), experts at the Waterloo Institute for Sustainable Aeronautics (WISA) are enlisting a host of new technologies to train more licensed pilots in less time, for less money and in a more environmentally friendly way.

At the heart of it all, WISA is utilizing AI and training machine learning algorithms to help instruct more pilots.

“The core idea of our research is to see how accurately AI could help assess pilot performance and support instructors in pilot training,” explains Dr. Shi Cao, a professor in Systems Design Engineering who leads the project. “The goal is not to replace humans with AI, but to assist and support humans with AI.”

Canada faces a significant pilot shortage, needing 7,300 more pilots by 2025, according to the Canadian Council for Aviation and Aerospace. Cao, an expert in human-factors engineering, believes solving this requires collaboration across industry, government and academia.

Today, pilot training programs blend traditional methods such as lectures, written materials and flying hours with new technologies.

“Flight simulators have become more common in recent years, creating opportunities to experiment with novel training methods. However, there is still much to explore around technologies such as data analytics, wearable sensors, virtual reality and augmented reality,” Cao says.

Cao and WISA colleagues Dr. Ewa Niechwiej-Szwedo, Dr. Elizabeth Irving and Dr. John Muñoz, are collaborating to show how new technologies can enhance pilot training, reduce environmental impact and deepen our understanding of pilot skill development.

To achieve their objectives, the team is first focusing on data collection, laying the foundation for their project by building an integrated data platform and developing data management strategies. They are advancing in creating image-recognition algorithms to precisely capture pilots' real-time actions, a feat not previously possible. Simultaneously, they are gathering extensive sample data from flight simulators and aircraft to enhance their data analytics. This data is used to train machine-learning algorithms. With substantial and reliable pilot data, they plan to use AI to evaluate and predict pilot performance. The goal is to incorporate these AI tools into pilot-training programs to measure their effectiveness.

“Findings from our research could support the development of tools to automatically assess pilot performance and provide feedback for improvement,” Cao says. “This could complement the flight instructor’s role and allow trainee self-assessment in flight simulators or on solo flights where currently it is difficult to assess and give feedback.”

Cao and his WISA colleagues are benefitting from support on many fronts. A 2013 graduate of Waterloo’s Science and Aviation program, and now a pilot for a major Canadian airline, Brad Moncion is completing a PhD as part of WISA’s Collaborative Aeronautics Program. He’s enrolled in the Aeronautics option of Systems Design Engineering with Cao as his supervisor.

As one of the few Waterloo PhD students with aviation industry flying experience, Moncion has frequently been consulted by Cao and the other researchers about their work. He also designed an assessment tool that is being used by the research group.

“I do think Dr. Cao’s research will help pilot training going forward,” Moncion says, citing the high cost of training new pilots and the limited capacity that make such training inaccessible for too many people. “Implementing evidence-based training – using technologies like video recordings, flight data parameters and eye-tracking devices to assist in this assessment – and conducting more training in simulators, or with virtual or augmented reality can address these barriers.”

One important corporate partner is AdHawk Microsystems, a company founded by Waterloo alumni and faculty, which invented a new type of low-power, research-grade eye tracker. The WISA team is using this technology to record their eye-movement data. At the same time, the researchers rent aircraft from the Brantford Flight Centre and hire its flight instructor services to collect data both from student and licensed pilots. This information is added to the computational models to further enhance their quality. 

The WISA researchers have made significant progress, but much work lies ahead. Cao mentions they will keep collecting pilot data, refining their models and testing accuracy. Their findings will be published in academic journals for peer review, and they will train graduate students in this research area to support the future aeronautics sector.

“We have all these new technological tools,” Cao says. “Why not use them? The preliminary evidence is supporting this approach to pilot training, and we’re going to do more.”

This research project was funded by a $296,500 Research-for-Impact contribution delivered through WISA. That funding is part of a $9.17 million investment by FedDev Ontario as part of its Aerospace Regional Recovery Initiative. In total, WISA supported 38 Research-for-Impact projects.