Researchers have created a new testing and training tool to inspire solutions to the particularly complex challenges of developing autonomous vehicles (AVs) capable of handling winter weather.
Released today, the free, open-source dataset - dubbed the Canadian Adverse Driving Conditions Dataset (CADC) - is a collaboration of research teams at Waterloo Engineering and the University of Toronto that are working to advance AV perception algorithms.
Their hope is that researchers around the world will utilize the dataset - essentially a collection of over 100 sophisticated, annotated video clips - to develop, test and improve their own software systems for AVs to be able to see and recognize their surroundings in nasty weather.
“We want to engage the research community to generate new ideas and enable innovation,” says Krzysztof Czarnecki, an electrical and computer engineering professor who leads the Waterloo team. “This is how you can solve really hard problems, the problems that are just too big for anyone to solve on their own.”
Krzysztof Czarnecki is director of the Waterloo Intelligent Systems Engineering Lab (WISE Lab).
Researchers partnered on the project with Scale, a San Francisco-based AI infrastructure company that labelled vehicles, pedestrians and cyclists in data from 33 kilometres of driving in all kinds of winter conditions in Waterloo Region.
The data was recorded in 2018 and 2019 using eight cameras, a 360-degree lidar scanner and a GPS tracker aboard Autonomoose, the highly automated Lincoln MKZ hybrid that has been under development at Waterloo for more than three years.
“We’re hoping that both industry and academia go nuts with it,” says Steven Waslander, a former Waterloo Engineering professor who now heads the Toronto Robotics and Artificial Intelligence Laboratory (TRAIL). “We want the world to be working on driving everywhere, and bad weather is a condition that is going to happen. We don’t want Canada to be 10 or 15 years behind simply because conditions can be a bit tougher up here.”
The first open-source tool of its kind, the dataset was created to give AV software developers a way to test how well their perception algorithms perform when everything looks a little different with snow on the ground and more falling from the sky.
“These differences mean a self-driving car might miss a vehicle, might miss a pedestrian,” says Czarnecki, director of the Waterloo Intelligent Systems Engineering Lab (WISE Lab). “Accurate perception is absolutely crucial. Without it, nothing else works.”
Coloured bounding boxes are used to identify vehicles, pedestrians and cyclists in a new dataset developed by researchers to test and train autonomous vehicles for winter driving.
The dataset also allows researchers to train deep-learning artificial intelligence (AI) algorithms to detect objects in adverse conditions by showing them numerous annotated and verified examples of them.
Next steps in the project include adding labels for other objects, including traffic signals and road features, to the video segments, which vary from about 15 seconds to a minute in length.
“Data is a critical bottleneck in current machine learning research,” says Alexandr Wang, founder and CEO of Scale. “Without reliable, high-quality data that captures the reality of driving in winter, it simply won’t be possible to build self-driving systems that work safely in these environments.”
In addition to the dataset, researchers have posted an academic article on the project on arXiv, with plans to seek publication in the International Journal of Robotics Research.
The Canadian Adverse Driving Conditions Dataset (CADC) is available online.
Krzysztof Czarnecki is director of the Waterloo Intelligent Systems Engineering Lab (WISE Lab).
