Modes of Automated Driving System Scenario Testing: Experience Report and Recommendations

TitleModes of Automated Driving System Scenario Testing: Experience Report and Recommendations
Publication TypeJournal Article
Year of Publication2020
AuthorsAntkiewicz, M., M. Kahn, M. Ala, K. Czarnecki, P. Wells, A. Acharya, and S. Beiker
JournalSAE Int. J. Adv. & Curr. Prac. in Mobility
Volume2
Issue5
Pagination2248-2266
Date Published04/2020
ISSN2641-9637
KeywordsAutomated Vehicles, Simulation and modeling, Systems engineering, Test procedures
Abstract

With the widespread development of automated driving systems (ADS), it is imperative that standardized testing methodologies be developed to assure safety and functionality. Scenario-based testing evaluates the behavior of an ADS-equipped subject vehicle (SV) in predefined driving scenarios. This paper compares four modes of performing such tests: closed-course testing with real actors, closed-course testing with surrogate actors, simulation testing, and closed-course testing with mixed reality. In a collaboration between the Waterloo Intelligent Systems Engineering (WISE) Lab and AAA, six automated driving scenario-based tests were executed on a closed-course, in simulation, and in mixed reality. These tests involved the University of Waterloo’s automated vehicle, dubbed the “UW Moose”, as the SV, as well as pedestrians, other vehicles, and road debris. Drawing on both data and the experience gained from executing these test scenarios, the paper reports on the advantages and disadvantages of the four scenario testing modes. It also discusses several possible implementations of mixed-reality scenario testing, including different strategies for data mixing. The paper closes with recommendations for choosing among the four modes.

URLhttps://saemobilus.sae.org/content/2020-01-1204
DOI10.4271/2020-01-1204
Original Publicationhttps://www.sae.org/publications/technical-papers/content/2020-01-1204/
Refereed DesignationRefereed
Full Text

WISE Lab logo

Opportunities

We are looking for postdocs and graduate students interested in working on all aspects of autonomous driving.

For more information, visit Open positions.