Optimization seminar-David Torregrosa Belén-Convergence of a proximal stochastic subgradient method under the Kurdyka-Lojasiewicz condition

Abstract:This talk presents a proximal stochastic subgradient method for minimizing the sum of an expected cost and a lower semicontinuous, prox-bounded function. We target a broad class of nonconvex integrands obeying a nonsmooth, localized variant of the descent lemma in the decision variable, which in particular covers smooth losses with Lipschitz gradient. At each iteration, the expected cost is replaced by a sample average that is progressively refined, and the proximal stepsize is selected by an Armijo-type line search enforcing a
sufficient decrease property up to stochastic errors induced by the sample-based approximation. This framework accommodates more general problem formulations than existing methods and our analysis yields convergence guarantees that, to the best of our knowledge, are new even in the smooth setting. Specifically, we establish almost sure convergence of the sequence of function values and stationarity of every accumulation point of the trajectories under the relaxed requirement that the sample-size sequence be merely nondecreasing and unbounded. Leveraging the Kurdyka-Lojasiewicz property, we further proof convergence of the whole trajectory to a single stationary point. Finally, for exponential-type desingularizing functions and polynomially growing sample sizes, we derive explicit polynomial convergence rates, up to logarithmic factor, for both the function values and the iterates. This is a joint work with Felipe Atenas, Pedro Pérez-Aros and Alejandro Jofré, from the University of Chile.