Teaching self-driving cars how to handle the worst of winter
Engineering researchers team up to create a new, open-source dataset to train and test all-important perception algorithms in nasty weather
Engineering researchers team up to create a new, open-source dataset to train and test all-important perception algorithms in nasty weatherBy Brian Caldwell Faculty of Engineering
Researchers have created a new testing and training tool to inspire solutions to the particularly complex challenges of developing autonomous vehicles (AVs) capable of handling winter weather.
Released today, the free, open-source dataset - dubbed the Canadian Adverse Driving Conditions Dataset (CADC) - is a collaboration of research teams at Waterloo Engineering and the University of Toronto that are working to advance AV perception algorithms.
Their hope is that researchers around the world will utilize the dataset - essentially a collection of over 100 sophisticated, annotated video clips - to develop, test and improve their own software systems for AVs to be able to see and recognize their surroundings in nasty weather.
“We want to engage the research community to generate new ideas and enable innovation,” says Krzysztof Czarnecki, an electrical and computer engineering professor who leads the Waterloo team. “This is how you can solve really hard problems, the problems that are just too big for anyone to solve on their own.”
Researchers partnered on the project with Scale, a San Francisco-based AI infrastructure company that labelled vehicles, pedestrians and cyclists in data from 33 kilometres of driving in all kinds of winter conditions in Waterloo Region.
The data was recorded in 2018 and 2019 using eight cameras, a 360-degree lidar scanner and a GPS tracker aboard Autonomoose, the highly automated Lincoln MKZ hybrid that has been under development at Waterloo for more than three years.
“We’re hoping that both industry and academia go nuts with it,” says Steven Waslander, a former Waterloo Engineering professor who now heads the Toronto Robotics and Artificial Intelligence Laboratory (TRAIL). “We want the world to be working on driving everywhere, and bad weather is a condition that is going to happen. We don’t want Canada to be 10 or 15 years behind simply because conditions can be a bit tougher up here.”
The first open-source tool of its kind, the dataset was created to give AV software developers a way to test how well their perception algorithms perform when everything looks a little different with snow on the ground and more falling from the sky.
“These differences mean a self-driving car might miss a vehicle, might miss a pedestrian,” says Czarnecki, director of the Waterloo Intelligent Systems Engineering Lab (WISE Lab). “Accurate perception is absolutely crucial. Without it, nothing else works.”
The dataset also allows researchers to train deep-learning artificial intelligence (AI) algorithms to detect objects in adverse conditions by showing them numerous annotated and verified examples of them.
Next steps in the project include adding labels for other objects, including traffic signals and road features, to the video segments, which vary from about 15 seconds to a minute in length.
“Data is a critical bottleneck in current machine learning research,” says Alexandr Wang, founder and CEO of Scale. “Without reliable, high-quality data that captures the reality of driving in winter, it simply won’t be possible to build self-driving systems that work safely in these environments.”
In addition to the dataset, researchers have posted an academic article on the project on arXiv, with plans to seek publication in the International Journal of Robotics Research.
The Canadian Adverse Driving Conditions Dataset (CADC) is available online.
Building an artificial brain that will one day replace human drivers is an incredibly complex technical challenge, say the researchers leading Waterloo’s autonomous vehicle project
Waterloo researchers expect to test an autonomous car for the first time on a public road anywhere in the country
Waterloo is one of just eight schools across North America invited to participate in the AutoDrive Challenge
The University of Waterloo acknowledges that much of our work takes place on the traditional territory of the Neutral, Anishinaabeg and Haudenosaunee peoples. Our main campus is situated on the Haldimand Tract, the land granted to the Six Nations that includes six miles on each side of the Grand River. Our active work toward reconciliation takes place across our campuses through research, learning, teaching, and community building, and is centralized within our Office of Indigenous Relations.