PhD Comprehensive Seminar | Yun Su, Deep neural network for solving Hamilton-Jacobi-Bellman equations

MC 6460 

Candidate

Yun Su | Applied Mathematics, University of Waterloo

Title

Deep neural network for solving Hamilton-Jacobi-Bellman equations

Abstract

Recently, the focus on optimal control issues has significantly increased in the fields of engineering and finance. The variational method is commonly used to transform these problems into Hamilton-Jacobi-Bellman (HJB) equations. Traditionally, HJB equations do not yield a classical solution, but they do provide weak solutions, known as viscosity solutions. The concept of a viscosity solution imposes a pointwise ordering condition that compares the candidate solution with smooth functions. Hence, for the numerical scheme to identify the correct viscosity solution, it needs to respect ordering in some sense. If a numerical scheme is monotone, pointwise consistent, stable, and ensures the existence of numerical solutions, then the numerical solutions will converge to the viscosity solution of the Hamilton-Jacobi-Bellman (HJB) equation.

However, neural networks struggle to capture these ordering properties when compared to numerical methods. Traditional neural networks use residual errors to guide the network towards the correct solution. Interestingly, we have found that many dissimilar functions still maintain a very low HJB-residual nearly everywhere. In the above circumstances, to produce Lipschitz continuous functions which satisfy (IVP) if the equation is understood in the "almost everywhere" sense. However, "generalized" solutions in this sense are not unique.

To overcome the limitations of traditional neural networks, our objective is to develop a novel neural network architecture that improves the convergence of the network towards the value function (viscosity solution) and  optimal control.