Waterloo Engineering researchers have designed smart technology to help industries navigate geopolitical uncertainty, inflation and supply chain issues.
Led by Dr. Luis Ricardez-Sandoval, a professor in the Department of Chemical Engineering, the team has harnessed machine learning (ML) to improve industrial production scheduling. This ability to improve planning helps manufacturers withstand unpredictable market conditions more effectively.
Using ML methods, the researchers have designed and built “smart agents” with special memory layers called long short-term memory (LSTM). These agents can remember events that took place in the past (historical data) and can make multiple decisions simultaneously.
The agents are trained through continuous simulations of plant processes that include unexpected events, for example, equipment failure or a sudden change in production demands. These are some of the variables that manufacturers must consider, which can fluctuate daily.
“We train the agents to provide scheduling decisions in real time that adjust to the uncertainty of the process, avoiding the need to solve complex optimization problems every time there is an unexpected event so we can ensure that we’re deploying an attractive production schedule in real-time,” Ricardez-Sandoval, a Canada Research Chair in Multiscale Modelling and Process Systems Engineering, said.
Imagine a chemical plant where a production manager must adjust and maximize daily production plans based on the arrival of supplies, how many processing units (reactors) are needed, optimal operating conditions for each unit and expected demands from the clients.
The research group employs Reinforcement Learning, a machine learning technique, to train the virtual agents to make real-time scheduling decisions, using feedback from their actions to promote learning. The training approach instructs them to focus on the main task even when conditions change or unexpected issues arise. Testing on two industrial plants concluded that integrating memory with Reinforcement Learning is effective in managing complex large-scale scheduling problems.
Dr. Luis Ricardez-Sandoval (right) with PhD student Daniel Rangel-Martinez (left).
“The agent's "brain” is designed using Artificial Neural Networks based on networks that are commonly used in fields like robotics and natural language processing. This helped us create a specialized agent that can effectively deal with the uncertainty found in industrial processes," Daniel Rangel-Martinez, a PhD student on the project, said.
When tested against an agent that did not have LSTM, the proposed method performed more efficiently. The next step is to investigate self-attention models which could accelerate training by focusing on the most critical information.
This research holds promise for use in real-world industrial settings where accuracy, flexibility and the ability to adapt to change are critical to ensure competitiveness and relevance.
A paper on their work titled Recurrent Reinforcement Learning Strategy with a Parameterized Agent for Online Scheduling of a State Task Network Under Uncertainty | Industrial & Engineering Chemistry Research was published in the journal of Industrial & Engineering Chemistry Research.