Continuous Optimization Seminar - Steve Vavasis

Title: Tutorial on back-propagation and automatic differentiation

Abstract: In this presentation, I'll cover the basics of automatic differentiation (AD). Then I'll explain how to apply AD for finding the gradient of one term of the loss function in training a neural network, which is called "back-propagation".  I'll also explain how some authors are using AD for the purpose of selecting hyperparameters in optimization algorithms.