WISE Automated Driving System and UW Moose


WISE Automated Driving System (ADS) is a self-driving car software stack developed by WISE Lab. WISE ADS is a derivative of Autonomoose. It consists of the following components in these three functional areas:

Sensor drivers

Object detection and tracking


  • Calibrations (publishes intrinsic camera calibrations and extrinsic sensor calibrations as static transforms)
  • Localizer (given GPS+IMU, wheel speeds publishes an optimal estimate of ego position and dynamic transforms @ 100 Hz)
  • Occupancy (lidar-based, publishes an occupancy grid @ 20 Hz and a ground grid @ 20 Hz. Additionally, publishes the ground and obstacle point cloud segments and ground normal vector @ 20 Hz) 
  • OpenPCDet-based 3D Object Detector (lidar-based, uses third-party PCDet PointPillars model, publishes 3D bounding boxes @ 20 Hz)
  • KF Tracker (tracks 3D bounding boxes, publishes object tracks and predictions @ 20 Hz)
  • Lane detector and projector (camera-based, uses third-party CondLaneNet, publishes camera images with overlaid detected lanes (2D) and detected lanes projected to vehicle ground plane (3D) @ 10 Hz)

A longer video showing perception and tracking from 3D perspective:

Remote video URL

A video showing lane detection and projection into vehicle ground plane in 3D:

Remote video URL

A video showing lidar point cloud segmentation done in Occupancy and the resulting occupancy and ground grids.

Remote video URL

A video showing Velodyne VLS-128 lidar point cloud projection into camera images.

Remote video URL


  • HD Map Server (loads Lanelet1 and Lanelet2 maps, publishes HD map message (once) and route through the map @ 1 Hz)
  • Behavior Planner + Rule Engine (given the location on the map, route, object tracks, rules of the road, publishes a maneuover to be executed and the global path with speed limits and lane boundaries @ 10 Hz)
  • Local Planner (plans an optimal route given a global path with speed limits and lane boundaries + the occupancy grid, publishes local path @ 10 Hz)


  • Vehicle Controller (given the local path, publishes throttle, steering, break, and gear commands to follow the path @ 50 Hz)


Operating System and Middleware

Wise ADS runs on Ubuntu 20.04 Focal and Robot Operating System (ROS) 1 Noetic.

Programming languages

Most components are implemented in C++ 17 and Python 3 with the exception of the Rule Engine, which is implemented as a NodeJS REST service.

UW Moose

UW Moose

WISE ADS is deployed on UW Moose, which is the University of Waterloo's self-driving test platform.

  • 2015 Lincoln MKZ with "technology package"
  • Dataspeed Drive-by-wire Interface, reports @ 100 Hz, commands @ 50 Hz.
  • Velodyne VLS-128 AlphaPrime lidar @ 20 Hz (gPTP synchronized)
    Velodyne VLP-32C lidar @ 20 Hz (PPS synchronized)
  • 8x Ximea HD cameras @ 10 Hz (hardware triggered)
  • Novatel Span ProPak6D GNSS receiver with NTRIP base station (~2 cm accuracy) @ 20 Hz and IMU @ 100 Hz
  • AMD Ryzen 9 7950X3D workstation with 1x NVidia RTX A6000 GPU, 96 Gb RAM, and 6Tb of NVMe SSD storage

Refer to the "Canadian Adverse Driving Conditions Dataset" paper for a recent detailed description before the addition of the VLS-128 lidar and the Ryzen 9 workstation.

UW Moose in front of AVRIL with VLS-128 in snow


WISE ADS can be used as the ADS with WISE Sim simulator.

AVRIL & AVRIL Base Station

UW Moose is located in Autonomous Vehicle Research and Intelligence Lab (AVRIL).

WISE Lab operates an AVRIL NTRIP base station necessary to achieving the high-precision (~2 cm accuracy) GNSS performance of UW Moose. The base station provides coverage within an area of ~40km around AVRIL allowing the UW Moose to operate in the Waterloo Region. The base station is open to the wider community.


Autonomoose, 2016-2019

Original Autonomoose team, mainly members of WAVE Lab led by prof. Waslander and WISE Lab led by prof. Krzysztof Czarnecki.

WISE Lab, 2019-present

Project Lead: prof. Krzysztof Czarnecki

Software and Hardware Maintainer: Dr. Michał Antkiewicz

  • Project management, release management, implementation, testing.
  • Dataspeed firmware upgrades.
  • NTRIP base station setup at AVRIL and on-board GNSS receiver reconfiguration.
  • Code migration from Ubuntu 16.04/ROS Kinetic to Ubuntu 18.04/ROS Melodic, and later to 20.04/ROS Noetic. Migration to Python 3.
  • Continuous integration and deployment using GitLab.
  • Lanelet2 mapping and map server support for Lanelet2.
  • WISE ADS mission setup using the same scenarios as in WISE Sim.
  • Behavior Planner Lanelet2 support.
  • System parameter tuning for optimal ADS operation and in response to functionality changes.
  • Occupancy grid node restructuring into three nodelets to improve performance: obstacle/ground pointcloud segmentation, occupancy grid using obstacle pointcloud segment, ground grid using ground pointcloud segment.
  • Improved lead-vehicle following in local planner.
  • Velodyne VLS-128 AlphaPrime lidar integration with gPTP time synchronization and updated calibrations.
  • Specification, procuring, and integration of the AMD workstation.
  • Two-stream (detector and simulator) bounding box mixing in tracker, tracker improvements.
  • New intrinsics (omnidir) for the cameras. New camera-to-camera and camera-to-lidar extrinsics. Perfect lidar into camera projection during motion when using motion correction.

Ground lidar mounting bracket CAD design and manufacture (stainless steel): Chris Tseng, May-Aug, 2023.

OpenPCDet-based object detector and tracker extensions: Adrian Chow, May-Dec 2021.

Lanelet2 maps, mapping tools, and map server extensions: Cameron Hinton, Jan-Apr, 2022.

Lane detector and projector, ground grid occupancy extension, Lanelet2 mapping tools: Joseph Younger, Jan-Apr, 2022.

Velodyne VLS-128 AlphaPrime lidar mounting plate fabrication, Chris Tseng, Jan-Feb 2023.