PhD comprehensive exam | Alex Howse, Optimization Methods for Tensor Decompositions

Optimization Methods for Tensor Decompositions

Abstract

Tensors, numerical multilinear arrays, are commonly used to organize and analyze data in applications throughout the sciences. Many of these applications make use of tensor decompositions, which express tensors as sums or products of several components, and can be used to obtain approximations in convenient formats. We consider the higher order singular value decomposition (HOSVD), which expresses a tensor as the multilinear product of a tensor with typically smaller dimensions and a set of matrices with useful orthonormality properties.

We present two optimization algorithms for computing HOSVD tensor approximations in a matrix manifold context: a nonlinearly preconditioned conjugate gradient (NPCG) iteration and a nonlinear GMRES iteration. In NPCG, a vector generated by a nonlinear preconditioner replaces the gradient in standard nonlinear conjugate gradient; and in nonlinear GMRES, a linear combination of past iterates and a tentative new iterate, generated by a nonlinear preconditioner, is minimized to produce an improved search direction. Numerical results show that these methods can significantly accelerate the standard iteration for computing HOSVD approximations.