Location
MC 6460
Candidate
Adil Attar | Applied Mathematics, University of Waterloo
Title
Stabilizing Natural Gradient Descent in Recurrent Neural Network Wavefunctions
Abstract
Understanding the ground states of quantum many-body systems is essential to characterising exotic phases of matter. While traditional quantum Monte Carlo (QMC) methods are powerful, they often fail when applied to quantum many-body systems with a sign problem. To complement these state-of-the-art methods, Neural Quantum States (NQS) have emerged as a promising alternative framework, which employ neural networks to learn the ground state wavefunction and its sign structure. Recurrent Neural Network (RNN) NQS leverage the autoregressive and generative capabilities of modern language models to learn wavefunctions efficiently. However, optimizing these architectures remains a challenge, as RNN NQS frequently suffer from severe training instabilities when paired with powerful second-order optimization techniques such as Stochastic Reconfiguration (SR) and its more efficient alternative minSR. We demonstrate that, with appropriate choices of hyperparameters and regularization, RNN optimizers can be trained successfully with these second-order methods and achieve several orders of magnitude better accuracy than standard first-order optimizers, namely Adam. Our findings provide a viable pathway to stabilize training, unlocking the full potential of high-order optimization for RNN wavefunctions.