Tackling malnutrition in older adults, bite by bite

Most older adults prefer to age in their homes and stay active in their communities. For many, malnutrition will derail those plans due to involuntary weight loss, micronutrient deficiencies and frailty. These conditions don’t just impact quality of life, they frequently necessitate a transition to living in a long-term care facility.  

To address this problem, University of Waterloo researchers are taking aim at malnutrition in aging populations with the development of an AI-powered application that can accurately measure people’s caloric and nutrient intake simply by “watching” what they eat. 

Unlike conventional nutrition apps that typically rely on the user to input information to make calculations and recommendations, the AI-powered app does all the work. The user doesn’t need to upload photos of their food, search a database for specific items or accurately measure and input how much of each item they’re eating.  

To use the app, you simply sit down at your table with a plate of food and a video camera mounted in a hardware device automatically detects when you start eating and records every bite that goes into your mouth. Behind the scenes, a visual language model (VLM) automatically analyzes the video, frame by frame, considering the utensil you’re using and the different stages of eating, including opening your mouth and chewing. Meta AI’s Segment Anything Model (SAM) keeps the nutrient analysis accurate by honing in on exactly what you’re eating and how much.  

“The more an app requires user input, the more likely it is that people won’t continue using it long term,” says Dr. Yuhao Chen, a research assistant professor in the Department of Systems Design Engineering’s Vision and Image Processing (VIP) Lab. “We want to support ongoing nutrition monitoring, so we’re using AI to automate processes and place fewer requirements on the user.” 

From presumption to precision 

The new system has been highly effective at analyzing the calories and nutrients of foods consumed using a spoon. The research is now expanding in scope to include more challenging utensils, like forks, chopsticks and even hands. 

The team also plans to go beyond analyzing food to incorporate other substances impacting a senior’s health and well-being. “The vision is to analyze whatever a person ingests, including supplements and drugs,” Chen says.   

No matter how the technology evolves, the aim remains to provide accurate caloric and nutrient tracking to older adults and their caretakers, make recommendations to fill identified nutritional gaps, and to do this in a way that is simple and user-friendly. 

“Some people aren’t comfortable using technology, so we want to make our application very easy to use,” says Chris Czarnecki (MASc ‘24), a recent master’s graduate who worked on the project. “It could take the form of a smartphone app that uses the device’s built-in camera, or even wearable glasses.  

“The goal is to help aging populations achieve optimal nutrition. Commercializing a solution that gives this growing demographic the tools to thrive while living independently in their own homes would be a huge success.” 

This AI-powered nutrition tracking system is an example of Waterloo’s innovative research on the determinants of health, including healthy aging. With the number of citizens aged 65 and older predicted to double over the next 30 years, widespread malnutrition in older adults has the potential to cripple long-term care and health-care systems.  

Funding for this research comes from National Research Council Canada’s Aging in Place Challenge program. It’s an initiative focused on improving the quality of life of older adults and their caregivers through innovation for safe and healthy aging.  

Leveraging diverse team skills to create real-world impact 

This research is the brainchild of Dr. Alexander Wong, co-director of the Vision and Image Processing (VIP) Research Group, and a professor in the Department of Systems Design Engineering at the University of Waterloo. Drs. Wong and Chen are collaborating with two experts in aging and nutrition, Drs. Heather Keller, a Schlegel Research Chair in Nutrition and Aging at the Research Institute for Aging (RIA), and Sharon Kirkpatrick, professor at Waterloo’s School of Public Health Sciences. Their work is supported by a team of graduate and undergraduate students spanning diverse areas of study at the University of Waterloo.