Advanced simulations may one day be able to help us explore new frontiers in atomic physics, build new materials and discover new drugs. But first, researchers must find the best ways to control these simulations. New research featured on the cover of Nature Machine Intelligence explores machine learning as a method for achieving optimal control.
Pooya Ronagh, Research Assistant Professor at IQC and the Department of Physics and Astronomy, demonstrated that a machine learning program can be trained to find the best ways to control systems it has never interacted with before.
“When people learn how to play one Atari game, we learn a lot about how to play every Atari game,” said Ronagh. “Right now, machine learning models need to be trained on each game separately. But what we really want is for them to be able to play the next game they have never seen before right off the bat.”
Ronagh’s team, which included Kyle Mills and Isaac Tamblyn of the Ontario Tech University and the National Research Council of Canada, simulated systems made up of particles. They wanted to control the temperature of these systems from hot—high energy—to cold—low energy, also known as the ground state. More specifically, they wanted to see if a type of machine learning called reinforcement learning could find the optimal schedule for dialing down the temperature in different system arrangements.
Reinforcement learning is a type of machine learning where a program learns how to best achieve a desired outcome by repeatedly interacting with a changing system. Regular temperature control schedules use analytic formulae. But the team found out that reinforcement learning can find highly irregular but better control schedules, especially when simulation is done on a much larger system.
The researchers’ program evolved a particle system from hot to cold repeatedly in a process of trial and error. After this period of training, the program was able to find improved control schedules on brand new systems it had never experienced. The program’s ability to find these schedules scaled well with the size of the system. Ronagh said that this performance is a promising start for using reinforcement learning for quantum control.
“The ultimate agenda is to build a good quantum computer, and for that, we need good qubits,” said Ronagh. “To build good qubits, you need to simulate the physics. You need to be able to control the evolution the system goes through.”
With its promising initial performance, Ronagh now hopes to extend the reinforcement learning technique to the quantum realm.
The team already started in this direction by making use of “destructive” measurements. To evolve the system from hot to cold, the control program needs to take temperature measurements along the way. However, taking measurements of a quantum system alters how it evolves. To test their program on quasi-quantum systems, the team instructed it to restart every time it took a measurement, emulating the destructive results of measuring a quantum system. They still achieved an improvement in scalability, suggesting reinforcement learning may be an effective method of quantum control.
And quantum control isn’t just useful for qubits. “At a higher level, many quantum algorithms simulate physical evolutions, and the performance of the algorithm depends upon the choice of these evolutions,” said Ronagh. “The reason drug discovery is so expensive is they have to make something, test it, and then throw it out and try a new mixture. What if I could simulate the drug and understand its properties with a quantum processor?”
To do that, researchers would need to run an algorithm that models the physics of the drug. “That in itself is an evolution. I must be able to control that evolution so that I get to the state of the drug that is the steady state seen in nature. The question is, what is the best way to achieve optimal quantum control? Machine learning is an exciting possibility.”
Finding the ground state of spin Hamiltonians with reinforcement learning was published in Nature Machine Intelligence September 7, 2020.