PhD Seminar • Theoretical Neuroscience • Foundations of Biologically Plausible Reinforcement Learning in Spiking Neural Networks

Please note: This PhD seminar will take place in E7 6323 and online.

Nicole Sandra-Yaffa Dumont, PhD candidate
David R. Cheriton School of Computer Science

Supervisors: Professors Jeff Orchard, Chris Eliasmith

Reinforcement learning (RL), particularly Temporal Difference (TD) learning, draws inspiration from and sheds light on the mechanisms behind animal learning. Despite their effectiveness, standard RL methods are still far from biologically realistic models of learning — they operate in discrete time, rely on perfect memory buffers, and use non-biological learning techniques like backpropagation.

In this seminar, we explore these issues and introduce new methods for biologically plausible RL. In particular, we introduce a novel TD learning rule for training spiking neural networks in continuous time using supervised Hebbian learning. Our method incorporates Legendre Delay Networks to emulate the functionality of time cells, offering a robust short-term memory system that can be used for updating value functions and successor features. Furthermore, we will explore biologically plausible approaches to policy learning and continuous action sampling in a simple spatial navigation task. This seminar presents a set of basic tools and models that offer fresh perspectives on modelling brain-like learning and decision-making processes.

To attend this PhD seminar in person, please go to E7 6323. You can also attend virtually using Zoom at https://uwaterloo.zoom.us/j/91655613525.