HOME /

PUBLICATIONS AND PROJECTS

PUBLICATIONS - Google Scholar

 

 

lightness 
Vision Systems For Identifying Interlocutor Behaviour And Augmenting Human-Robot Interaction - Winner of the Best Imaging Paper Award
  • Computer vision based ROI detection, feature extraction and processing
  • Video annotation and supervised signal detection, algorithm development and evaluation

 

 

  

Natural head and body orientation for humanoid robots during conversations with moving human partners through motion capture analysis - Best Paper Finalist Award
  • Vicon motion capture data collection and processing for unique HRI scenarios
  • Dynamic model formulation, non-linear least squares optimization, evaluation and analysis
  • Model deployment on REEM-C Humanoid Robot for replicating natural tracking behaviour

     
beampattern metric 
Estimating speaker direction on a humanoid robot with binaural acoustic signals - PLOS ONE 2024
  • Acoustical data collection and annotation in real-life scenarios on the REEM-C
  • Novel cost function formulations for speech detection, DOA estimation and real-time latencies in noisy robotic environments
  • Bayesian optimization for efficient parameter search and evaluation against traditional approaches

 

track gif track 
An Audio-Video Sensor Fusion Framework To Augment Humanoid Capabilities For Human-Robot Interaction - Humanoids 2023

 

  • Sensor fusion approach with acoustic and visual subject DOA measurements applying adaptive weightings and Kalman filter approach
  • Algorithm development for allowing REEM-C to redirect attention to human subjects in wide-ranged workspaces outside its FOV
  • Full ROS integration of microphones, camera and robot + Vicon motion capture and user study analysis and evaluation

 

 

PROJECTS AND AWARDS

 

Building Real Time Video AI Applications

  • Using NVIDIA TAO and NVIDIA DeepStream to build video AI pipelines for object detection and to improve performance with fine-tuning, pruning and quantization-aware training
  • Detected occurrences of tailgating in feed with NVIDIA DashCamNet (including use of ngc cli, full training config pipeline and DeepStream application architecture)

    detection
    infer


 

 

Robotics Algorithms

  • Extended Kalman Filter for LiDAR and RADAR fused localization of self-driving car
  • PID based reference trajectory following, tuning via Twiddle and Bayesian search
  • MPC prototyping for simulating car following a curved track, with throttle and steering based control

    ekf

 

 

Acoustic Event Detection

pred

  • Domestic acoustic event data collection (typing, rolling chair, clapping etc)
  • Transfer learning and fine-tuning ResNet-50 (among other models) for spectrogram based event classification
  • Experimenting with image based feature extraction using dilation/erosion image channels added to standard STFT output

 

Winner of ASA Challenge 2023 - Link to challenge

 

  • Won ASA Student Challenge, applying DOA measurement techniques and geometric approximations to estimate a diver's breathing rate, position and swimming speed
     

Second place, AI against COVID-19: Screening X-ray Images for COVID-19 Infections (Waterloo Kids) - Link to challenge

 

 

 

 

 

  • Built competitive deep learning pipelines for xray image classification (feature extraction methods, image processing, ResNet architecture tuning and training)
     

Winner of Citadel Datathon - Media

missing

  • Provided insights on smart-city planning using expansive dataset and proposed solutions for potential issues threatening civilian safety and city efficiency