Seminar by Krishnakumar Balasubramanian

Gaussian random field approximation for wide neural networks

Neural networks (NNs) with randomly initialized weights tend to have be a limiting Gaussian behavior at the random field level. There has been a flurry of recent work making this observation precise.  This talk will discuss bounds on Gaussian random field approximation of wide random neural networks of any depth, assuming Lipschitz activation functions. The bounds are on a Wasserstein transport distance in function space equipped with a strong (supremum) metric, and are explicit in the widths of the layers and natural parameters such as moments of the weights. The result follows from a general approximation result using Stein's method, combined with a novel Gaussian smoothing technique for random fields, which will also be described. The talk covers joint works with Larry Goldstein, Nathan Ross, and Adil Salim.