Title: Tutorial on back-propagation and automatic differentiation
|University of Waterloo
Abstract: In this presentation, I'll cover the basics of automatic differentiation (AD). Then I'll explain how to apply AD for finding the gradient of one term of the loss function in training a neural network, which is called "back-propagation". I'll also explain how some authors are using AD for the purpose of selecting hyperparameters in optimization algorithms.