PhD Comprehensive Exam | Guilherme Macieira De Araujo, Parallel-in-time methods for speeding up optimization algorithms

MC 5479

Candidate

Guilherme Macieira De Araujo | Applied Mathematics, University of Waterloo

Title

Parallel-in-time methods for speeding up optimization algorithms

Abstract

Parallel-in-time methods such as multigrid-reduction-in-time (MGRIT) and parareal utilize an iterative multilevel multigrid structure to solve linear systems of differential equations by solving a coarse-grid problem, which is less costly while also approximating the original, fine-grid problem. Such methods have been successfully applied to speeding up the solution of parabolic PDEs.

Various iterative optimization algorithms for solving unconstrained minimization problems, such as gradient descent and Nesterov accelerated gradient, take the form of one- or two-step updates. The convergence rates of these algorithms can be extremely slow and, fortuitously, solving them in parallel is equivalent to solving the same exact kind of linear systems to which MGRIT can be applied to. We also note that, for sufficiently small time steps, these algorithms can also be interpreted as discretizations of certain ODEs.

Inspired by these observations, we explore using parallel-in-time methods to parallelize discrete optimization algorithms in order to reduce the wall-clock time required to run them. We present exploratory results and analysis for both smooth optimization problems, with classical gradient and proximal methods, and nonsmooth optimization problems, with classical splitting algorithms, also suggesting pathways for future research explorations.